Table of contents

1.Introduction

Purpose and scope    

2.Background to the intervention

Description of the intervention and its objectives    

Baseline and points of comparison    

3.Implementation / state of Play

4.Method

4.1.Method for the evaluation support study

4.2.Internet based public consultation

4.3.European Economic and Social Committee Report

4.4.Limitations and robustness of findings

5.Analysis and answers to the evaluation questions

5.1.Effectiveness

5.1.1.Axis 1. Improving the competitiveness of the agricultural and forestry sector

5.1.2.Axis 2. Environment and the countryside

5.1.3.Axis 3. Improving the quality of life in rural areas and encouraging diversification of the rural economy

5.1.4.Axis 4. Building local capacity (LEADER)

5.2.Efficiency

5.2.1.Qualitative assessment

5.2.2.Quantitative assessment

5.3.Coherence

5.3.1. Coherence of rural development policy with other CAP funds and other EU interventions in the same programme area    

5.3.2. Coherence with the overall rural development priorities    

5.4.Relevance

5.5.EU Added Value

6.Conclusions

Annex 1. Lead DG, Decide Planning/CWP references

Annex 2. Stakeholder Consultation

Annex 3. Data on implementation of rural development programmes by measure

Annex 4. Intervention logic by axis

Glossary

1.Introduction

Purpose and scope

As part of the implementation of the 2007-2013 Rural Development policy, in 2015 Member States had to carry out the ex post evaluation of their rural development programmes. 1 These evaluations had to examine the degree of utilisation of resources, the effectiveness and efficiency of the programming of the European Agricultural Fund for Rural Development (EAFRD), its socio-economic impact and its impact on the Community priorities. They also had to cover the goals of the programme and aimed to draw lessons concerning rural development policy. They were expected to identify the factors that contributed to the success or failure of the programmes’ implementation, including as regards sustainability, and identify best practice.

The current evaluation provides a summary of the findings of these ex post evaluations and analyses the effectiveness, efficiency, coherence, relevance and EU added value of the 2007-2013 Rural Development Policy based on a synthesis of the work of the Member States. It builds on the ex post evaluations of the rural development programmes in 26 Member States: Croatia did not have an EAFRD financed rural development programme in the 2007-2013 period. Bulgaria did not provide an ex post evaluation of its rural development programme for the period in question during the preparation of the evaluation support study 2 .

A number of the issues raised in this evaluation had already been identified during the implementation of the programmes and were addressed in the legal framework for the 2014-2020 programming period (for example, revising the Common Monitoring and Evaluation Framework to improve the intervention logic, reduce the number of indicators and simplify the evaluation questions, and streamline reporting). The lessons drawn from this synthesis have further served to inform the preparation of the post 2020 Commission proposals for the modernisation of the Common Agricultural Policy (CAP) and will be useful for the assessment of the Member States’ CAP Strategic Plans that are central in the Commission’s proposals 3 .

The conclusions of this evaluation is also very relevant for the long-term vision for the EU’s rural areas 4 , which aims to enable rural areas to make the most of their potential and support them in facing their own unique set of issues, from demographic change to connectivity, the risk of poverty and limited access to services.

2.Background to the intervention

Description of the intervention and its objectives

The EU’s rural development policy aims at helping the rural areas of the EU to meet a wide range of economic, environmental and social challenges. Frequently called "the second pillar” of the Common Agricultural Policy (CAP), it complements the system of direct payments to farmers and the measures to manage agricultural markets (the so-called "first pillar"). EU rural development policy offers a flexible approach, based on the principles of subsidiarity and partnership. When designing rural development programmes, Member States have a significant degree of flexibility in finding a balance between the sectoral dimension (i.e. agricultural restructuring) and the territorial dimension (i.e. land management and the socio-economic development of rural areas). From a menu of approximately 40 support measures, Member States select those best suited to address the specific strengths and weaknesses of individual programming areas.

In a first step, based on a thorough analysis of the economic, social and environmental situation of their rural areas and their potential for development, Member States submitted national strategy plans translating to the Member State situation the EU priorities agreed in the Community strategic guidelines for rural development 5 . These priorities were: improving the competitiveness of the agricultural and forestry sector; improving the environment and the countryside; improving the quality of life in rural areas and encouraging diversification of the rural economy; building local capacity for employment and diversification; ensuring consistency in programming (maximise synergies between axes); and complementarity between Community instruments. 

In a second step, based on a SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis, choosing from a set of measures for which they could receive Community financial support, Member States established, at national or regional level, integrated rural development programmes, specifying what funding was to be spent on which measures in the period 2007-2013. Member States and regions had to spread their rural development funding between three thematic axes: Axis 1 - economic concerns (competitiveness of agriculture and forestry), Axis 2 - the environment and the countryside (biodiversity, climate change, sustainable resource use in agriculture and forests) and Axis 3 - social aspects (quality of life). Moreover, a part of the funding had to support the implementation of local development strategies by the local action groups of LEADER (Liaison Entre Actions de Développement de l’Économie Rurale). This "LEADER approach" to rural development involves individual projects designed and executed by local partnerships to address specific local problems in line with a local development strategy (LDS). This horizontal approach was implemented through Axis 4 of the Rural Development.

The whole process of preparing the rural development programmes was carried out in close cooperation between the Member States and the Commission services.

Funding from the European Agricultural Fund for Rural Development (EAFRD) of each rural development programme had to be spread between the four axes, as a safeguard to ensure that the programme reflected the main policy objectives. The required minimum funding was 10% for Axis 1, 25% for Axis 2, 10% for Axis 3 and 5% for LEADER (2.5% for the new Member States). These minimum percentages were sufficiently low to leave Member States or regions a high margin of flexibility to emphasize the policy axis in function of their situation and needs. The EAFRD had a total endowment of EUR 96.2 billion for the 2007-2013 period 6 .

Table 1. Overview of rural development measures

Source: DG Agriculture and Rural Development

The figure below presents a high-level intervention logic for the 2007-2013 rural development programmes. A more detailed intervention logic per axis is presented in the Annex 3.

Figure 1. Intervention logic for the 2007-2013 rural development programmes

Source: DG Agriculture and Rural Development

In 2008, a CAP policy review process, known as the ‘health check’, included an increase in modulation, whereby direct payments to farmers were reduced and the funding transferred to rural development (Council Regulations (EC) No. 73/2009 and (EC) No. 74/2009). In addition, a further EUR 1.02 billion was made available to Member States and regions to reinforce their rural development programmes as part of the European Economic Recovery Plan agreed in December 2008. The additional funding was reserved for actions related to climate change, renewable energy, water management, biodiversity, innovation linked to these four areas, developing broadband internet in rural areas, or for accompanying measures in the dairy sector.

An important element of the 2007-2013 rural development policy was the creation of a common monitoring and evaluation framework, including an indicator set (output, result, impact and context indicators) and periodic evaluations to be carried out by the Member States on the basis of predefined evaluation questions.

The ex post evaluation of a rural development programme is the final part of ongoing evaluation of the programming period of 2007-2013, built upon its monitoring and evaluation activities conducted throughout the programme implementation period.

Baseline and points of comparison

In 2006, which served as baseline for the ex post evaluations, the EU-27 7 rural areas (predominantly rural and intermediate regions) represented 91% of the territory and 56% of the population 8 . The corresponding shares for predominantly rural areas were 54.4% of the territory and 19.2% of the population. Rural areas are therefore particularly important in terms of territory. Among the Member States, the importance of rural areas varied from the more "urban" ones (Belgium, Netherlands, Malta) to the more "rural" ones (Ireland, Finland, Slovenia) along a continuum where intermediate regions can play a major role. Even if economic activity tends to be concentrated in more densely populated areas, rural areas in 2006 generated 43% of the Gross Value Added (GVA) in the EU-27 and provided almost 55% of the employment 9 . However, compared to urban areas, rural areas lagged behind as regards a number of socio-economic indicators: demographic situation in some areas, income per capita, lower employment rate and higher unemployment rate, human capital, activity of women and young people, development of the tertiary sector as well as other aspects linked to the quality of life.

The impact assessment in view of the post 2006 rural development policy 10 carried out in 2004 and the ex-ante evaluations 11 pointed to the difficulties of many rural areas to adjust to changes brought about by a combination of factors, including continuing technological change, a decline in real food prices and in the proportion of income spent on food, and the difficulties of competing in increasingly global food markets.

As a result, at the time, rural regions faced significant economic problems, including:

·high rates of unemployment, which stood at 11.1% in predominantly rural areas, compared to an average of 9.8% across the EU-25 as a whole (1999/2001 average);

·widespread problems of underemployment;

·low levels of Gross Domestic Product (GDP) per head, which in predominantly rural areas stood at 71% of the EU-25 average in 1999-2001; and

· a relative shortage of new development opportunities in many rural areas.

At the start of the programming period, Member States were at different levels of dependence on the primary sector. Within the EU-27, employment in the primary sector in 2006 was 6%, yet with wide variations, from 1.5% in the UK and 1.6% in Luxembourg to 20.5% in Bulgaria and 30% in Romania. The central and eastern European Member States were particularly heavily dependent on agriculture, however, also here significant variations are apparent, less than 4% of the workforce of Czechia and Slovakia were employed in agriculture in 2006 12 .

In terms of gross value added, there were also large variations: the primary sector accounted for 0.4% of total GVA in Luxembourg, but for 8.8% in Romania in 2006 13 .

At the social level, the more peripheral local areas were characterised by depopulation, caused by negative rates of change in natural population (where there are more deaths than births) and high rates of out-migration (especially among young people), leaving behind an ageing population. Rural population is usually older than average.

At the same time, many less remote rural areas in Europe faced very different issues, as they often presented an attractive alternative to living in cities, and were experiencing population growth. In many Member States, in-migration to rural areas from urban areas was an increasing trend, often involving increased populations of commuters and retired people. This led to other problems such as increased development pressures in the countryside, a change in the balance of rural communities, and shortages of affordable housing as wealthier urban incomers out-bid local residents.

The 2004 impact assessment also pointed out that Europe’s rural environment was subject to a variety of pressures, stemming from the intensification of land management in many areas, and its abandonment in others.

The marginal viability of both agriculture and forestry has brought a widespread risk of land abandonment to many rural areas with consequences for the landscape and environment. Agriculture and forestry are the main land users and play a key role in the management of the natural resources in rural areas and in shaping rural landscapes and wildlife habitats. Managed farmland and forests make an important contribution to maintenance of high natural value areas in the EU.

3.Implementation / state of Play

The 2007-2013 programming period was fully closed by 2020. The overall financial implementation rate for the 92 programmes for the 2007-2013 period reached 97.0%, where EUR 93.4 billion funds were consumed in comparison with the EUR 96.2 billion of initially available EU funds.

Figure 2. EAFRD expenditure by Member State (in millions)

Source: DG Agriculture and Rural Development

Overall, the largest share of EAFRD spending went to Axis 2 (Environment), followed by Axis 1 (Competitiveness). These two account for above 80% of EAFRD spending. The differences in Member States choices are illustrated in Figure 4 .

Figure 3. Share of EAFRD spending by axis

Source: DG Agriculture and Rural Development

Figure 4. EAFRD spending by Axis per Member State

Source: DG Agriculture and Rural Development    

Figure 5 (below) presents an overview of the most important measures in terms of share of the EAFRD contribution for the EU-27.

Figure 5. EAFRD measures by importance

Source: DG Agriculture and Rural Development

Overall, the rural development programmes in the programming period contributed among other to:

·578 000 farm modernisation projects;

·start-up support for 192 000 young farmers;

·385 000 farmers' involvement in quality schemes;

·more than 6.8 million training days for farmers;

·More than 48 000 000 physical hectares supported under agri-environment.

4.Method

The purpose of this evaluation is to provide a synthesis and an analysis of the ex post evaluations of the 2007-2013 rural development programmes submitted by the Managing Authorities, to identify common trends, highlight differences between the programmes and provide conclusions and lessons learned. This evaluation is based on three main sources of input:

·The evaluation support study of the external contractor. This is based on the evaluations carried out by the Member States 14 .

·A public consultation that ran from 22 January 2018 to 20 April 2018 and obtained 720 replies.

·The European Economic and Social Committee (EESC) information report ‘Ex post evaluation of rural development programmes 2007-2013’ 15 .

Additionally, findings published in Special Reports of the European Court of Auditors were used where relevant for this evaluation.

4.1.Method for the evaluation support study

The evaluation support study is based on the Member States’ ex post evaluations. These were carried out by either external or in some cases in-house, but always functionally independent evaluators, based on a set of evaluation questions predefined at EU level. The Commission provided guidance on intervention logic, evaluation questions, indicators and evaluation methods via the publication of guidance documents 16 , the setting up of an evaluation helpdesk 17 and by providing a forum for the exchange of good practices via the creation of the Expert Committee on Evaluation of rural development programmes. Despite these efforts, the quality of the ex post evaluation reports were varying from one programme to another.

As a first step, the evaluation support study had to provide a synthetic answer to a set of evaluation questions based on the replies given by each individual Member State to exactly the same set of standardised questions in their ex post evaluation of the rural development programmes.

These questions were either:

·Programme related, assessing the effectiveness of the programme in reaching its objectives and the efficiency of the rural development programmes as a whole (‘programme related questions’),

·Measure related, assessing the effectiveness of the individual measures in the rural development programmes (‘measure related’).

Additionally, a few questions addressed specifically the implementation and effectiveness of LEADER.

Member States’ evaluators could select the evaluation methodology they found best suited, allowing them to take into account local circumstances.

The Member States’ ex post evaluation reports contained both quantitative and qualitative information on target and implementation values (output and result indicators). However, the quantitative data could not be used to provide an overall judgment on the effectiveness of the implemented rural development programmes, as target values were adjusted multiple times during the life cycle of the programme. Moreover some of the adjustments made to the target values consisted of unnecessary reductions, resulting in multiple cases where ex post evaluations observed a degree of implementation (comparing implemented to target values) above 100%, sometimes even largely above this figure. Therefore the qualitative data provided in the ex post evaluation reports bore particular importance in the overall assessment of both the single measures and programmes performance.

Therefore, the programme-, measure- and LEADER-related questions were assessed using three criteria:

1)Extent (of contribution):

For each of the questions, the Member States’ evaluations were categorised by the contractor on the basis of the quantitative and particularly qualitative data provided in the ex post evaluation reports in relation to the measures’ impact as follows:

- positive contribution;

- limited contribution;

- no contribution;

- negative contribution;

- not clear;

- not measured.

Based on the share of reports that reported a positive conclusion over all those that provided a conclusion for the question at stake (so excluding those categorised in “not clear” or “not measured”), the extent was judged as

- very limited if the share is 25% or lower;

- limited if the share is 26-49%;

- medium if the share is 50-75%;

- high if the share is above 75%.

2)Plausibility:

This measures if the judgement made on extent of contribution is based on sufficient evidence. Plausibility of the answers was based on the share of the ex post evaluation reports that provided a conclusion. Reports for which the contribution was not clear or not measured were considered as not having provided a conclusion.

- very plausible: >85% of the reports provide a conclusion;

- plausible: 50-85% of the reports provide a conclusion;

- not plausible <50% of the reports provide a conclusion.

3)Certainty:

Finally, for the programme related questions, certainty judges the quantitative assessment based on the completeness and comparability of the quantitative data:

- certain: >80% of the regions/MS have comparable indicators;

- partly certain: between 50-80% have comparable indicators;

- not certain: <50% have comparable indicators.

As a second step, the support study provided answers to eight synthesis questions. These covered the evaluation dimensions of effectiveness, efficiency, relevance, coherence and EU value added of the Rural Development Policy. The answers to these synthesis questions are based on the relevant programme, measure and LEADER evaluation questions in the Member States evaluations, as well as other sources of information such as input, output and result indicators.

For each synthesis question, the relevant programme and measure related evaluation questions were identified. Then, using each time the answers to the relevant (programme and measure) questions, the number of answers indicating negative contribution is deducted from the number of reports stating positive contribution. The number of reports providing limited contribution have a score of 0.5 while reports indicating positive contribution have a score 1. All contributions (positive, limited and negative) are divided by the total number of reports used. For the programmer-related questions the total number is 91 while for the measure-related questions the total number is equal to the number of regions or Member States that implemented the measure. The calculation of the score follows the following formula:

E = (P - N + C*L) / R 18

The system leads to a final score that can reach values between 1 and 0; these are labelled as follows:

4.2.internet based public consultation

Another source of information was an internet-based public consultation. This was online from 22 January until 20 April 2018 and generated 720 answers; the questionnaire contained open and closed questions and allowed for the upload of documents. After checking for campaigns, the questionnaire was analysed using the classical statistical tools (frequency tables, cross-tables). The results of the public consultation (see Annex 2. Stakeholder Consultation for a more detailed analysis of the public consultation) were used in this staff working document to corroborate the findings of the evaluation support study in particular as regards the answers to the synthesis questions.

4.3.European Economic and Social Committee Report

A final source of information, as mentioned in the methodology, was the EESC information report on the “Ex post evaluation of 2007-2013 rural development programmes” 19 . This aimed at assessing the effectiveness, relevance and EU value added of the measures in the 2007-2013 rural development programmes. This report, which from the onset was designed by the EESC as a complement to the Commission’s evaluation, focussed on three questions:

·What were the main shortcomings of the rural development programmer 2007-2013?

·What was the main added value of the rural development programmer 2007-2013?

·To what extent and how was civil society included in the rural development programmer 2007-2013?

The report was based on:

·The information collected during fact-finding missions in 4 Member States 20 . During the missions, the Members conducted semi-structured interviews civil society organisations to gather evidence of the practical experiences with the rural development programme.

·A consultation (online questionnaire) of organisations in 13 Member States 21 on their views concerning the strengths and weaknesses of the programme implementation. The questionnaire was targeted at organisations representing aggregate interests rather than at individual citizens.

4.4.Limitations and robustness of findings

The rural development policy was designed 17 years ago, well before the current better regulation context and its performance management. This meant in particular that the issues such as baseline, targeting and indicators (definition and data quality) and intervention logic had received considerably lower attention than nowadays.

The evaluation was largely based on a synthesis of substantive results coming from the Member States’ ex post evaluations. Notwithstanding the help offered by the Commission to Member States in preparing the evaluations, a number of difficulties and limitations made it de facto very challenging to reliably assess the substantive results:

·Lack of capacity for evaluation and lack of harmonised evaluation methodologies used by the evaluators in the different Member States/regions;

·The uneven quality of and different level of detail in the different evaluations;

·The incompleteness or implausible values of some of the indicator data;

·The ex post evaluation reports’ quantitative information on target and implementation values (output and result indicators) could not be used to provide an overall judgment about the effectiveness of the implemented rural development programmes, as target values were adjusted multiple times during the life cycle of the programme. Moreover, some of the adjustments made to the target values consisted of unnecessary reductions, resulting in multiple cases where ex post evaluations observed a degree of implementation (comparing implemented to target values) above 100%, sometimes even largely above this figure. Therefore the multiple adjustments of targets during the programme implementation made the use of target indicators unreliable for benchmarking.

These limitations led the external evaluator responsible for the support study to base the judgements largely on the approach explained above. The support study nevertheless presents quantitative information using result and impact indicators, but as explained by the evaluator in the report these should rather be considered as illustrations.

This approach has its limitations: as it mechanistically compares the positive cases over the total, it cannot take into account the extent of the realisations of the rural development programmes; neither does it take into account whether or not the Member States’ evaluation could establish a causal relationship between the intervention and the result.

Despite these limitations, it is reassuring that the overall findings of the support study, the public consultation and the fact-finding done by the EESC are confirming each other in broad terms.

5.Analysis and answers to the evaluation questions

As indicated in the evaluation roadmap, the evaluation conclusions are drawn on a summary of the replies to the complete set of revised 22 common evaluation questions 23  of the 2007-2013 ex post evaluations carried out by the Member States and the set of evaluation questions.

This entails that the evaluation support study treats in total 53 evaluation questions, most of them further broken down in sub-questions.

The report is supported by drawing analysis from the EESC report as well as from internal DG Agriculture and Rural Development data and other EU Commission data sources (Eurostat).

5.1.Effectiveness

Effectiveness is measured through the achievement of the objectives and the extent to which it is due to the rural development programmes.

The results, and the assessment of the impact made by the rural development programmes described in this Staff Working Document have to be considered in the context of the overall economic, social and environmental situation and trends in the rural areas, which were in the middle of the programming period also hit by the economic crisis. The December 2013 Rural Development Report 24 presents a comprehensive set of data describing the situation at the end of 2013. For later years, information is published online 25 . The following information is derived from these sources, unless otherwise indicated.

5.1.1.Axis 1. Improving the competitiveness of the agricultural and forestry sector    

Axis I measures aimed to boost modernisation and competitiveness in the agricultural sector, which would lead to economic development. Particularly investments aiming to improve productivity, efficiency and sustainability are key to increase GVA in the primary sector while also creating positive spill-over effects on the economy. Axis 1 also targeted employment in agriculture, and promoting knowledge transfer and training, which led to the employment of young people in sectors other than the primary sector.

5.1.1.1 Productivity

Agricultural labour productivity increased drastically during the programming period (+48%), from EUR 11 887 per annual work unit (AWU) in 2006 to EUR 17 585 /AWU in 2015. Moreover, total factor productivity grew by 8.5% over the same period ( Figure 6 ). However, there still are important disparities between Member States: in 2015 labour productivity in agriculture varied from EUR 4 015 /AWU in Latvia to EUR 68 402 /AWU in the Netherlands. Besides differences in level, there are also strong differences between Member States in terms of development: while in Estonia productivity more than doubled to 12 345 EUR /AWU in 2015, in the same timespan it decreased in Malta from EUR 14 309 /AWU to EUR11 612 /AWU 26 .

The increase in productivity was driven by the replacement of human capital by machinery. It was thus accompanied by a decrease in the total number of agricultural holdings, from 13.8 million in 2007 to 10.8 million in 2013 27 , pointing towards a concentration of UAA (Utilised Agriculture Areas) in larger farms (those holdings of 100 ha or more) 28 .

Over the same period, factor income per AWU 29 increased by 39% in real terms. Income in agriculture increased faster in the Member States which joined the EU after 2004, in line with the strong productivity gains after accession, supported by the rural development policy (151% average increase).

Figure 6. Total factor productivity in agriculture in EU-27 30

Source: DG Agriculture and Rural Development

Member States have spent EUR 30 billion in measures aiming at improving competitiveness mainly through investments in farm modernisation (M121), adding value to agricultural and forestry products (M123), and improving infrastructure in rural areas related to the development and adaptation of agriculture (M125) ( Figure 7 ).

A majority (59%) of the Member States/regions stated in their ex post evaluation reports that the effects of the policy on competitiveness were positive. These effects can be attributed mainly to trainings and investments in modern machinery and technology.

This is confirmed by the results of the public consultation, distinguishing clearly between the positive impact on competitiveness in agriculture, which is judged more favourable than in the ex post evaluations with 74% of positive views 31 and forestry, with not as strong positive views (44% of positive views 32 ). There were no major difference of opinion between stakeholder groups.

Figure 7. Expenditure by measure in Axis 1

Note: M111 corresponds to vocational training and information actions; M112, setting up of young farmers; M113, early retirement of farmers and farm workers; M121, modernisation of farms; M123, adding value to agricultural and forestry products, M125, improving infrastructure related to the development and adaptation of agriculture and forestry

Source: DG Agriculture and Rural Development

Some rural development programmes attempted to quantify the impact of the rural development policy on competitiveness in the agricultural and forestry sector. Based on these figures, the support study estimated that EU labour productivity increased by 4.1% thanks to the rural development policy (the median was +3.2%) and concluded that the objective of higher competitiveness was achieved to a moderate extent. On the other hand, the support study could not conclude on the contribution of the rural development policy to improved competitiveness in the overall rural economy, due to the limited number of reports (18%) including data result indicators and the high variability of the results (-6% to 17% ). Many rural development measures have a significant impact on the competitiveness of the agricultural and forestry sector.

M121 (modernisation of farms) had a total expenditure of EUR 11.7 billion, implemented by 27 Member States across 88 regions. The main aim of the measure was to modernise the technology, management and practices that are applied in agricultural holdings. This measure supported 467 324 investment operations in the equivalent of 4% of EU holdings 33 . The overall investments led to an increase in GVA in the supported holdings of EUR 19.3 billion (11.1% of the agricultural sector GVA). These results were confirmed by the qualitative analysis, explaining that the competitiveness was increased in multiple ways: 39% of the ex post evaluations found an increase of GVA, 31% an increase of labour efficiency, 21% the introduction of new or better products, and 21% the introduction of new technologies. Furthermore, of those reports that provided a conclusion on M121 (86% of the total), 83% stated a positive contribution 34 . Thus, the support study concluded that the measure contributed to an improved competitiveness to a great extent.

Another significant measure is M123 (adding value to agricultural and forestry products), with a total level of expenditure of EUR 4 413 million, supporting 28 265 enterprises. The aim of the measure was to modernise operations, introduce innovative procedures and promote the integration of value chain segments to increase the value added of agricultural and forestry products. It was implemented by 25 Member States in 85 regions and contributed towards two result indicators, one of which is the increase in GVA in supported enterprises that equaled EUR 30 billion (17.4% of the GVA of the agricultural sector in 2015). The other result indicator ‘number of holdings with new products and/or techniques introduced under M123’ amounted to 14 484 holdings supported under this measure (0.1% of the total agricultural holdings in the EU). The qualitative analysis confirms this result, as 76% of the reports 35 found that the measure had a positive contribution to competitiveness. The support study concluded that this measure contributed to a great extent to an improved competitiveness.

In addition, under M125 (improving infrastructure related to the development and adaptation of agriculture and forestry) the investments were aimed at roads and irrigation systems. The measure was implemented in 24 Member States with 71% of the reports that provided a conclusion on it stating a positive contribution 36 . Out of all the reports signalling a positive effect on increasing competitiveness, 27% attributed it to improved road structure and access, which resulted in reduced costs of transportation along the value chain and 26% attributed it to the new irrigation structures, which significantly reduced the cost of water. Overall, M125 supported 56 779 operations and generated a value added of EUR 7 billion, making it one of the most successful measures in generating value added, 4% of the total GVA for the agricultural sector in the implementation period.

M113 (early retirement of farmers and farm workers) promotes competitiveness by supporting farmers and farm workers with early retirement, allowing access for younger farmers to farms and farmland. It allows young farmers to profit from economies of scale through the expansion of their company, and/or improving old farms through the introduction of modern farming techniques. EUR 2.7 billion was spent on Measure 113 by 16 Member States across 51 regions, resulting in an output of 28 398 farmers and 231 farm workers with early retirement, and 493 516 hectares of land released. Out of the total number of farmers, 0.1% has enjoyed early retirement under M113. Out of the total amount of arable land in the EU, 0.3% was released. Several case studies, like Andalusia and Aragón, showed that the early retirement led to an increase in competitiveness in the agricultural sector. 83% of the rather small number of ex post evaluation reports 37  analysing the measure found that if contributed positively to improving the competitiveness of the beneficiaries.

Although the evaluation support study found that the measure contributed to an improved competitiveness to a great extent, it also concluded, along with the Evaluation on Generational Renewal 38 that there is very weak or no correlation between support for early retirement (M113) and the change in the number of young farm managers. The ratio of young farmers per elderly farmers did not change between 2007 and 2013 and stayed at 19 young farmers (under 40) per 100 elderly farmers (those older than 65) 39 . The measure was also discontinued in the current programming period, as it was not suited to older farmers’ specific needs and in addition, the measure was criticised also by the European Court of Auditors 40  for being cost-inefficient.

Young farmers stimulate competitiveness as they are more inclined to modernise farms, introduce new methods and techniques, and start innovative processes. Thus, it was the aim of M112 (setting up of young farmers) to set up young farmers, either through supporting the creation of a self-owned business, or through employment in already existing businesses. It was implemented by 24 Member States across 69 regions 41 , with a total expenditure of EUR 3.3 billion. The total number of farm managers under the age of 40 supported to set-up was 192 003. Of those reports that provided a conclusion on M112 (59), 78% stated a positive contribution on competitiveness from the increase of young farmers in agriculture, producing an increase of gross value added of EUR 6 131.8 million in these Member States/regions (3.6% of the GVA in the agricultural sector in the EU-28 in 2015 level). Therefore, the support study concluded that the measure contributed to a great extent to an improved competitiveness.

A total budget of EUR 840.5 million was spent on M111 (vocational training and information actions) by 26 Member States across 75 regions 42 . Outputs of 6.4 million participants in training and 6.8 million days of training were reported, representing 29% of active farm workers in the EU as a whole. However, this comparison is not straight forward as the same farmers may have participated several times in training activities. Furthermore, the result indicators for this measure show that, out of the 6.4 million participants, 1.86 passed trainings by achieving a certificate, degree or diploma (on average 80 971 per MS) and 1 987 552 implemented the achieved skills (on average 132 503 per MS). The support study concluded that this measure contributed to a moderate extent to an improved competitiveness.

Member States also assessed the impact of rural development programmes on milk, which is presented in detail in Box 1 .

5.1.1.2 Employment in agriculture

During the programming period, employment in the primary sector has further decreased. In 2016, the farming sector was employing 9.5 million workers, a figure 25% below 2007 43 .

This decline is, nevertheless, overestimated, because of changes in survey thresholds (mostly between 2007 and 2010) that excluded some farms (e.g. those below 1 ha in Poland) from the statistics. The decrease in farm labour was greater in those Member States that joined the EU in or after 2004 (EU-N13) than in the older Member States (EU-15) 44 , especially over the period 2005-2011. This reflects structural changes after accession to the EU. The greatest reductions in the number of farm workers took place in Bulgaria throughout the period, while in Slovakia, Romania, Estonia and Latvia in particular between 2005 and 2011.

At the end of the implementation period, in 2015, agriculture still accounted for 4.17% of employment in the EU-28, down from 4.85% in 2008; for forestry these figures were stable at 0.25% 45 .

Member States/regions reported in the ex post evaluation the number of jobs created thanks to the rural development policy under result indicators RI8.1 (gross number of jobs created in agriculture) and RI8.2 (gross number of jobs created in forestry). They amounted to respectively 1.1% and 1.0% of the total number of people working in agriculture and forestry.

As regards the employment of young farmers, both the evaluation support study and the Evaluation Staff Working Document on Generational Renewal 46 , (as confirmed by the CAP Performance report 47 ) found that the generational-renewal measures (direct payments, setting-up and investment support) do not influence significantly the number of young farmers across the EU as a whole. They mainly target the maintenance or protection of farming jobs, particularly in marginal areas, rather than the creation of new farming jobs. It is also important that support be delivered in a targeted way with advice and training to improve the quality of those jobs and the performance of supported businesses. In particular, the World Bank report Thinking CAP 48 highlighted that the CAP supported the creation of better (i.e. more remunerative) jobs for the workers who remained behind in agriculture, while it improved quality of life and reduced poverty in agricultural areas.

The JRC analysis on the CAP’s impact 49 also concluded that all CAP support and particularly CMO and direct payments had a large causal impact on safeguarding employment in agriculture.

5.1.2.Axis 2. Environment and the countryside

The status of natural resources deteriorated over the programming period. Nevertheless, Member States/regions reported positive contributions from the rural development policy, suggesting that the deterioration of natural resources would have been worse without the policy, notably because of land abandonment. On this basis, the evaluation support study concluded that the rural development programmes contributed to better water management, to a moderate extent to the protection of natural resources and landscape and to climate change mitigation to a low or unclear extent. Regarding the supply of renewable energy, despite the positive impact, the extent of it was difficult to determine and quantify.

The EESC report on the “Ex post evaluation of rural development programmes 2007-2013” 50 also concluded that rural development programmes played a big role in promoting more environmentally-friendly production methods in agriculture. The public consultation also pointed to the positive impact on the environment: (75% positive views 51 ) and on the countryside (74% positive views), with no major difference between stakeholders’ opinions.

Over the programming period, Member States spent EUR 44.7 billion on environmental measures. The share of agricultural area under beneficial farm practices supported by the CAP was: 46% for biodiversity, 30% for water quality, 35% for soil, 30% for water quality and 49% for the avoidance of marginalisation and land abandonment.

Member States spent EUR 11 billion in measures aiming to improve quality of life in rural areas and encourage diversification of the rural economy, mainly through agri-environment payments (M214), payments to farmers in areas with handicaps other than mountain areas (M212) and payments to farmers in mountain areas (M212).

Figure 9. Expenditure by measure in Axis 2

Note: M211 corresponds to natural handicap payments to farmers in mountain areas; M212 payments to farmers in areas with handicaps other than mountain areas; M213, Natura 2000 payments and payments linked to Directive 2000/60/EC; M214, agri-environment payments; M215, animal welfare payments; M221, first afforestation of agricultural land; M226, restoring forestry potential and introducing prevention actions

Source: DG Agriculture and Rural Development

5.1.2.1.Protection and enhancement of environment, landscape and forestry

Over the programming period, the Farmland Bird Index continued its downward trend (except for forest species). The share of estimated agricultural area affected by moderate to severe water erosion (>11 t/ha/year) stood at 6.7% of total agricultural area in 2010. The soil mean organic content decreased slightly from 43.28 g/kg (in 2009) to 43.10 g/kg (in 2015). By contrast, nitrates surplus, affecting water quality, reduced on average by 15.8 kg N/ha/year. In addition, the area under organic farming increased by 10% from 2012 to 2015 to 6.2% of the UAA.

In 2015, EU-28 UAA stood at 179 million hectares, 3.7% below the 2006 baseline. Permanent grassland and meadow, key for carbon sequestration and biodiversity, remained stable at 33.8% 52 of UAA. The share of the different land cover categories varies across the EU and is correlated with the physical characteristics of the territory such as mountains and remoteness of the area 53 . Nevertheless, the total area used for extensive grazing increased significantly between 2007 and 2013, from 20.6% of the total UAA in 2007 to 29.4% in 2013 54 .

61% of the ex post evaluations reported positive effects on the protection and enhancement of natural resources and landscape, particularly biodiversity and High Nature Value farming and forestry, through measures preventing land abandonment 55 . The programmes were either said to improve the situation or prevent further degradation.

Figure 10. Evolution of the farmland bird index (2005-2019)

Source: DG Agriculture and Rural Development on the basis of Agridata (CTX_ENV_35_1)

M214 (Agri-environment payments) was of key importance with respect to protecting natural resources and biodiversity, as all programmes have implemented this measure, and it accounted for the largest proportion of expenditure (26%). Almost 90% of reports giving a clear judgement on this measure concluded that its implementation made a positive contribution to the environment, through both the extent of area covered by commitments and their effectiveness. Biodiversity and High Nature Value farming were identified as most significantly positively affected (69%), improved soil quality in 66% and water quality in 61% of reports.

Overall, Member States spent EUR 23.6 billion on different operations aimed at improving the environmental situation. Most regions have implemented a variety of sub-measures with different focuses, which involved more than 1.5 million farms (14% of the total) and more than 12 000 other land managers. The measure concerned 48 million hectares and 0.97 million livestock units (0.7% of the total livestock units in the EU 56 ); as well as:

-42.5 million ha under land management beneficial for biodiversity. Organic farming and extensive cultivation methods (including the renunciation of biocides), especially extensive grassland management, are favourable cultivation methods for the promotion of biodiversity. Furthermore, positive impacts on biodiversity were explicitly attributed to the promotion of beekeeping and to the support of the use of indigenous breeds at risk of extinction on livestock farms, among others.

-35.7 million ha under land management beneficial for water quality (see I mprovement of water managem ent , below, for more detailed information),

-36 million ha under land management beneficial for soil quality. The effects were mainly attributed to the introduction of organic farming and livestock and extensive grassland management, though only a few partial measures are aimed directly at soil protection. However, all contractual nature conservation measures as well as measures with the main objective of improving water quality contributed to the protection or improvement of soil.

-19.6 million ha under commitments to avoid marginalisation and land abandonment. Rural development programmes shown that organic farming, extensive grassland management and contractual nature conservation measures in particular prevent marginalisation and land abandonment. Nevertheless, although most of programmes reported on the impact of this measure, only 25% of the evaluation reports rated its contribution to avoidance of marginalisation and land abandonment as positive.

Based on these findings the evaluation support study concluded that the measure contributed to the improvement of the environmental situation to a high extent.

Some of the other most relevant measures under Axis 2 were M211 (natural handicap payments to farmers in mountain area) and M212 (payments to farmers in areas with handicaps, other than mountain areas). They compensate farmers for the additional costs and income losses arising from the difficulties of agricultural production in areas with handicap. Their core objective is preventing land abandonment, to which they contributed positively for 60% of the Member States/regions. These measures were implemented in 25 Member States and covered 52.8 million ha.

M211 was implemented in 15 Member States in a total of 60 regions, with a total budget of EUR 7 391.1 million. The measure resulted in 1 million (9.7% of the total agricultural holdings in 2013) supported farm holdings and 16 million hectares covered (one Member State implementing the measure did not report the output indicators). This amounted to 57% of the UAA in the less-favoured mountain areas of the Member States that implemented the measure and to 56.7% of the UAA in mountainous areas in the EU as a whole. Over two thirds of the reports stated that it made a positive contribution to the environment (especially biodiversity and high nature value farming/forestry), mainly through maintaining extensive agricultural systems in areas where appropriate land management is important for biodiversity and High Nature Value habitats. It was also recognised as helping retain population and employment in fragile rural areas. The support study concluded that the measure contributed to an improvement of the environmental situation to a high extent.

M212 (Payments to farmers in areas with handicaps, other than mountain areas) was implemented in 27 Member States in 75 regions, with a total budget of EUR 7 681 million. On average, Member States spent 10% of their total EAFRD expenditure in Measure 212, with wide disparities. This resulted in an output of 1.8  million supported farm holdings (17% of the total) and 36.8 million hectares covered, corresponding to 59% of the UAA in the less-favoured areas other than mountain areas of the Member States that have offered the measure.

The majority of the ex post evaluation reports that assessed it, confirmed the measure’s positive impact on improving the environment through prevention of land abandonment 57 . The positive impact is mainly attributed to its support to maintain agricultural cultivation on sites, which otherwise would have been threatened by land abandonment and a possible degradation of natural values. Some reports also stated that the measure promoted extensive farming practices.

Both measures’ main environmental effect was on biodiversity and high nature value farming/forestry, with 28% of the Member States and regions quoting a positive impact on biodiversity, while 14% a rather limited impact, 17% no contribution, 10% unclear (14% did not assess this aspect). This is due to the fact that the maintenance of local agricultural techniques is a basis for the conservation of animal and plant communities dependent on this management and that the measure covers ecologically valuable areas. Additional positive effects resulting from the implementation of these measures on other natural resources (soil, water) and climate mitigation are possible but not plausible according to the analysis in the evaluation support study.

Another significant measure was M213 (Natura 2000 payments and payments linked to Directive 2000/60/EC), which with a total expenditure of EUR 267.5 million supported farmers’ incomes in more than 75 000 supported farm holdings (0.7% of the total EU farms), in 13 Member States that implemented it on 1.1 million hectares (28 rural development programmers, 15.3% of agricultural area in Natura 2000 of the Member States that have offered the measure). It compensated them for financial losses due to using different farming practices beneficial for the designated habitats/species. Whilst the management practices are compulsory, even without these payments, positive effects on biodiversity were reported due to improved understanding and acceptance of Natura 2000 requirements and high coverage of target areas. The actual effects on the environmental aspects were not clear (on soil, mitigation and adaptation of climate change and prevention of land abandonment) except for biodiversity, which was generally considered somewhat more positive than the overall environmental impact. In 28% of the ex post evaluation reports the general contribution of M213 to improving the environment was assessed as positive, while 31% stated a limited effect and 17% saw no contribution. 10% of the evaluations of the measure indicated that the effect was not clear (and 14% did not carry-out the assessment). The support study concluded that the measure contributed to an improvement of the environmental situation to a limited extent.

20 Member States spent EUR 1.6 billion on M221 (first afforestation of agricultural land) and reported on it in 63 ex post reports 58 . In addition to the afforestation of agricultural land and the expansion of the entire forest area, the measure also pursued the goal of introducing sustainable forest management in these areas. This measure reached a total output of 203 944 hectares of afforested land in 42 531 supported actions, representing 0.12% of the total forest and wooden lands (FOWL) 59 . Nevertheless, to the improvement of the environmental situation mostly regarding biodiversity, mitigation of climate change and water quality. This was largely attributed to the diverse and various ecological functions of forests and hence the afforested areas were often assessed positively without further consideration.

Another measure under this axis was M226 (restoring forestry potential and introducing prevention actions), with a total expenditure of EUR 1 542.1 million, was assessed as highly positive for the environment, mostly through preventing fires and flooding. These actions, implemented by 16 Member States across 56 regions, supported 10 million hectares of damaged forests (for a total of 77 359 actions) 60 . In 78% of the positive cases the improvement of the environmental situation was attributed to the fields of fire damages and prevention. Improved water quality and flood mediation and biodiversity were also pointed at. On the basis, the evaluation support study concluded that the measure contributed to the improvement of the environmental situation to a medium extent.

5.1.2.2.Climate change mitigation, adaptation and renewable energy

In 2015, agricultural emissions of greenhouse gases (GHG) in the European Union amounted to 494 million tons of CO2 equivalents. This accounted for about 12% of total EU-28 emissions for that year 61 . While total net emissions from agriculture declined by 3% compared to 2006 baseline, the share of agriculture in GHG emissions showed an increase over the last years. This is due to the decrease in total GHG emissions in the overall EU economy, greater than the decrease of agricultural emissions.

Figure 11. Greenhouse gas emissions from agriculture

Source: DG Agriculture and Rural Development (IMP_07_2 and IMP_07_1)

The production of renewable energy from agriculture increased by 600% between 2006 and 2015, to 292 million MWh 62 .

Only a reduced number of Member States (7) reported quantitative figures on energy production and the reported production capacity installed thanks to rural development support was small (614 779 MWh for the 2007-2013 period). Therefore the contribution of rural development programmes towards the supply of renewable energy (on just the basis of energy production) is not certain.

Despite the data limitations, the qualitative data shows that the rural development programmes positively contributed to the supply of renewable energy, but only to a very limited extent (in 12% of the Member States/regions). In the majority of these regions, this positive development is based on the support for biogas and biomass energy. In some regions, investments in solar energy and distribution systems contributed to an expansion of renewable energies. For another 42% of reports, positive effects were reported, however with an unclear extent. Limited effects were reported for 26% of the regions.

Although renewable energy production was not the specific focus of any rural development measures, the ex post evaluations reported on it in relation to measure M114 (use of advisory services by farmers and forest holders) and M311 (diversification to non-agricultural activities). The ex post evaluation reports indicated that Member States or regions enhanced the transition from fossil sources to renewable biomass based sources, installed renewable energy production plants or generally produced energy from renewable sources through M311.

As regards climate change mitigation and adaptation, overall only 6% of the Member States/regions reported on the environmental measures’ positive effects in relation to mitigation, which were however seen rather as a side effect than as a result of clear targeting in the rural development programmes. Positive effects, however with an unclear extent, are found in 69% of regions. For another 19% limited effects of climate change mitigation and adaptation are stated.

The following measures were identified as positively contributing: forest-related measures like afforestation and forest management, agri-environmental measures and measures concerning energy-savings and the production of renewable energies.

Member states reported in their rural development programmes a reduction of GHG emissions of 742 849 tons CO2eq per year, thanks to the rural development policy.. Compared to agricultural emissions, the reduction represents 0.15% of the agricultural emissions in 2015 63 . The reported level of carbon sequestration sums up to 349 387 Mega tons CO2eq/a, resulting from measures on woodland and agricultural land. Compared to the assumed overall potential of carbon sinks in forests and croplands in the EU, the rural development programmes’ contribution accounts for a share of 0.08%. The total area under land management contributing to mitigating climate change supported by the rural development programmes amounted to 4 813 km² (0.15% of the total area of agriculture and forestry in the EU-28, 3 220 159 km²).

The support study concluded that M214 (Agri-environmental payments) had a positive impact on climate protection through the reduction of greenhouse gas emissions, but the effect was very limited, only indirect and difficult to measure, given especially that climate protection was not the focus of these measures, although M214 covered 25.5 million hectares with measures contributing to mitigating climate change.

M221 (first afforestation of agricultural land) also positively contributed to climate mitigation, according to the ex post evaluations. Through site-specific afforestation and sustainable use, forests can be established as a long-term carbon sink. The contribution of the measure to mitigating climate change was assessed as predominantly positive (46% of the reports). Particular emphasis was placed on the positive effects on CO2 storage function in biomass and soil of the newly established forest stands (73%), but also the long duration and slow process should be kept in mind (12%). Afforestation further reduces greenhouse gas emissions as it generally introduces a more extensive management on the area (23%).

Another measure with positive effect was M215 (animal welfare) and it was also explicitly mentioned in the ex post evaluation reports through the consolidation of environment-friendly farming practices, related to extensification and low livestock density.

Most of the other measures under this axis also showed to have effect on climate mitigation, but based on the analysis made by the contractor and the low number of reporting Member States, climate mitigation was achieved as an indirect effect of these measures.

5.1.2.3.Improvement of water management

The total irrigated area barely increased from 2007 to 2013, from 173.7 million hectares to 174.6 million 64 hectares. In 2012, the share of monitoring sites of nitrates in groundwater with a poor quality stood at 11.7%% and 14.2% with moderate quality, and for surface water at 11.4% and 31.7% respectively 65 . In 2013 the pressure from agriculture was decreasing, although not uniformly, in terms of numbers of animals and the consumption of inorganic fertilisers, continuing a long-term trend 66 .

The ex post evaluation reports do not use the term “water management” in a uniform way, leading to a very different approach to the evaluation of use, quality and quantity of water. Furthermore, ex post evaluations very rarely reported on those issues 67 and with a significant lack of consistency in the use of measurements. Therefore, the effects of the CAP on water use and water quantity could only be assessed qualitatively and with difficulty. Nevertheless, overall more than half of the ex post evaluation reports indicated that the rural development programmes positively contributed to water management. Out of 91 Member States/regions, 64% suggest positive impact on water management, and 16% limited or no impact. 20% of evaluations did not provide information or stressed that such an impact could not be assessed based on the available data.

As it was mentioned before, M214 (Agri-environmental payments) was a key measure also for water quality improvement. The area under successful land management contributing to water quality was 35.7 million hectares in 27 Member States, making it the most widespread measure in regard to water quality. 61% of the ex post evaluation reports found positive contributions of this measure to the quality of groundwater and surface water, although the protection of water resources was not emphasized in all sub-measures. M214 was focused on the reduction of nitrogen input into groundwater, but reports also described indirect positive impacts in the reduction of nutrient and pollutant inputs in surface waters (phosphorus). In addition to the reduction of fertilizer and pesticides, the most successful instruments were promoting adapted soil cultivation, permanent covering/vegetation strips, low-emission spreading technology, preservation of semi-natural pasture habitats, retention dikes. 18% of the ex post evaluation reports attested that M214 had a limited contribution to water quality, mainly because the effects were only indirect and difficult to assess. No contribution to water quality was seen in 5% of the evaluated reports, 10% could not clearly identify a contribution.

An indirect contribution to water management came from M121 (modernisation of agricultural holdings), as it fomented innovation, and 27% of the ex post evaluations referred specifically to environmental benefits coming mostly from water management and green technology, and another 16% to environmental benefits coming from innovations. Another measure that the ex post evaluation reports highlighted as contributing positively towards better water management, was M125 (improving infrastructure related to the development and adaptation of agriculture and forestry), by promoting investments in newer irrigation systems that allowed for better use of water resources, reducing the overall water consumption.

The direct impact of many other relevant measures in water management, even when potentially positive, was deemed as not possible to substantiate by the analysis made by the contractor for measures such as support to areas with handicaps, mostly due to the very low number of Member States/regions reporting on water management and water quality, and low number of clear positive impacts in the ex post evaluations.

5.1.3.Axis 3. Improving the quality of life in rural areas and encouraging diversification of the rural economy

Member States spent EUR 11 billion in measures aiming to improve quality of life in rural areas and encourage diversification of the rural economy, mainly through support to investments in basic services for the economy and rural population (M321) and for village renewal and development (M322). Relatively few rural development programmes contributed to both aspects of Axis 3 and a larger number contributed to either quality of life or diversification.

Figure 12. Expenditure per measure under axis 3

Note: M311 corresponds to diversification into non-agricultural activities; M312 to support for business creation and development; M313 to encouragement of tourism activities; M321 to basic services for the economy and rural population; M322 to village renewal and development; M323 to conservation and upgrading of the rural heritage; M331, training and information; M341 to skills-acquisition and animation measure with a view to preparing and implementing a local development strategy.

Source: DG Agriculture and rural development

Out of the four objectives of rural development policy, this is the objective for which the evaluation support study concluded to the most limited degree of achievement, but it is also the one for which there are fewer measures with a direct effect and the lowest evidence in Member States/regions reports, as well as low quality data.

5.1.3.1.Growth and diversification of the rural economy

The importance of the primary sector in the total economy (EU-28) has slightly decreased from 1.66% of total GVA in 2006 to 1.6% in 2016 68 . Rural areas’ importance in the overall economy also decreased, representing 9.5% in 2006 to 12% in 2015.

Roughly one third of all EU farmers (34%) were engaged in gainful activities other than their farm work in 2010. For the vast majority of farmers engaged in other gainful activities, these were the main source of income, according to the support study.

The importance of the tertiary sector (services) in employment increased during the period 2006-2015 while the one of the primary sector decreased further. In 2015, it accounted for only 4.7% of total employment (from 5.8% in 2006 69 ). In rural areas it declined from 15.7% to 11.8%, in intermediate areas from 6.5% to 5% and in urban from 1.2% to 1% 70 .

Figure 13. Structure of employment by branch

Note: Primary sector covers agricultural production, forestry, fishing and mining; the secondary sector refers to manufacturing, the tertiary sector are services.

Source: DG Agriculture and Rural Development on the basis of Eurostat

The impact on the quality of life was commented on in 47 ex post evaluation reports. The positive effects can be mainly attributed to basic infrastructure services (e.g. mobility and communication) and the social dimension (such as governance, social sustainability).

The impact on the diversification of the rural economy was commented on in 37 ex post evaluation reports. The non-agricultural sector most often mentioned was tourism.

Only half of the ex post evaluation reports provided qualitative data regarding the contribution of the rural development programmes to the growth of the whole rural economy. The qualitative data provided in the rural development programmes show that the policy had positive effects on economic growth in 57% of the Member States/regions. These effects can be attributed mainly to investments in modernisation and labour productivity, as well as to investments in human capital. Although 20% of the reports found that the rural development programmes had limited effects on the growth of the rural economy, at least 9 evaluations noted that they have mitigated the effects of economic recession in a significant way, by limiting the loss of jobs and the decrease in the number of farms.

Based on the limited quantitative overview, the contribution of the rural development programmes to improving the quality of life in rural areas and encouraging diversification of the rural economy is not certain. However, the qualitative overview indicates that the rural development programmes have contributed to a very limited extent.

One of the measures impacting most economic diversification has been M311 (diversification to non-agricultural activities). This measure was implemented by 20 Member States across 65 regions and a total budget of EUR 1.1 billion. This resulted in a total of 41 940 beneficiaries, though amounting to less than 0.5% of the total number of holdings in the EU. Investment on this measure increased GVA in supported businesses by EUR 563.6 million, and created 27 881 jobs. In addition, 52% of all the ex post evaluations found the measure to have a positive impact on improving the economic diversification of the beneficiaries, contributing to the stabilisation of farms, enabling farmers to increase their overall performance, assisting farm households to maintain or increase their income, supporting farms to maintain employment or even to create new jobs.

In order to improve the quality of life in rural areas, EUR 3.7 billion was spent by 23 Member States across 70 regions on M321 (basic services for the economy and rural population). This produced an output of 63 215 supported actions, and in the Member States that have offered the measure, there has been one action per 1 298 person (1 545 at fully EU level) in relation to the total population of the rural area. Furthermore, more than 45 million persons benefitted from improved services in rural areas and more than 16 million persons benefitted from improved internet coverage in rural areas The positive impact was confirmed 58% of the qualitative ex post evaluation reports, which found that the measure positively contributed to improving the quality of life of the beneficiaries. Life quality improvements came from social infrastructure investments (within the thematic complex of day-care, schools, families, community facilities as well as health), as well as further internet access (see   rural development p rogramme ’s contribution to the creation of access to broadband internet for more detailed information on the topic). Furthermore, waste/drinking water infrastructure, electricity and heat were named, as well as the field of transport and roads, as being improved by this measure. In terms of the sectors in which the quality of life was improved, the focus was on the ‘(public) services’ sector in general (64%) and on the social dimension (16%).

M322 (Village renewal and development) was implemented by 50 regions in 22 Member States, with a total spending of EUR 3.2 billion. Its objective was to counteracting population decline by making villages attractive places to live and work, and promoting business activity. The implementation of this measure resulted in 41 577 villages were actions took place for 40 million persons. In relation to the total population of the rural area, in the Member States that have offered the measure, there has been one action per 1 651 person. The ex post evaluation reports confirmed that the measure contributed positively to improving the quality of life of the beneficiaries in 77% of the Member States which reported on this measure. The improvement can be mainly attributed to intangible aspects (such as social cohesion, political participation, collective ownership in shaping the community environment, or local/regional identity), infrastructure (such as meeting places), the attractiveness of villages for housing and technical and traffic infrastructure (including waste/drinking water infrastructure, roads and other public spaces).

5.1.3.2.Job creation in Rural Areas

The rural employment rate for the age group from 15-64 year olds slightly increased, from 63.3% in 2006 to 65.0% in 2015 halving the gap with the overall employment figures which stood at 64.3% in 2006 and 65.6% in 2015.

The quantitative data shows that the rural development programmes contributed to create more than 159 000 new jobs, not just in the primary sector but also in the entire economy. It should be mentioned that the validity of these results could be diminished as only 45% 71 of the reports provided data on the number of jobs provided and it does not account for the support to generational renewal and to the maintenance of farmers in agriculture.

While the largest share of new jobs was created in the agricultural sector, most reports concluded that the programmes helped the creation of new jobs in other sectors, particularly food and tourism, but also culture and services.

The qualitative data shows that 48% of the rural development programmes had a positive effect on the creation of new jobs, which is reflected by the quantitative results too. Furthermore, the reports highlighted the important role of the rural development programmes in limiting the effects of the economic recession on the job market, and in preserving existing jobs.

Even when the creation of employment happened mostly in the agricultural sector, reports acknowledged that rural development programmers also led to the creation of new jobs in other sectors and especially food and tourism. The creation of employment prospects, especially for young generations in rural areas also limited the abandonment of farms, notably in mountainous regions.

M322 (village renewal and development) was identified as having a positive impact for non-beneficiaries as well as beneficiaries of the support, and in 85% of the ex post evaluation reports, additional economic effects were named (mainly jobs and diversification). M313 (encouragement of tourism activities) allowed for the creation of tourism actions (24 518 created) and jobs 17 578 in 22 Member States that reported. Other measures, like M323 (conservation and upgrading of the rural heritage) or M321 (Basic services for the economy and rural population) also indirectly impacted job creation in rural areas, though they did not have associated job related result indicators, and therefore their impact cannot be quantified.

The European Economic and Social Committee showed in its report that rural development programmes contributed to maintain agriculture and promoted competitiveness of rural businesses and employment and diversified rural economies. The public consultation pointed also to a positive impact on the quality of life: (75% a positive contribution) and the countryside (74% a positive contribution), with no major difference between stakeholder groups for none of the questions. These results contrast with the qualitative data stemming from the ex post evaluation reports, where only 18% of the 91 Member States/regions identified a positive impact for both domains (i.e. quality of life and diversification) and 9% identified a limited impact.

The recent JRC study on the effects of the CAP proved the positive effect of the CAP in general and of rural development policy in particular (see Box 2) with regard to not only safeguarding employment in agriculture, but also creating jobs in rural areas.

5.1.3.3.Introduction of innovative approaches

The rural development programmes helped introducing innovation across different areas in 55% of the target regions through modernising infrastructures and introducing new technologies and products, as well as by introducing changes in behaviours and processes, especially thanks to LEADER solutions and concepts. However, not all measures contributed to introducing innovative approaches and 26 reports found that rural development programmes had a small or no impact on innovation. The fact that the programmes had a different way of defining innovation posed some challenges to the achievement and measurement of such results. In addition, several reports pointed out that innovation was not a specific objective of the policy.

From a quantitative perspective, the contribution of the policy to the introduction of innovative approaches was small, 33 027 holdings introduced innovative products and techniques thanks to CAP support, which only accounted for 0.3% of the total agriculture holdings in the EU in 2013. Besides, there were very large differences registered across the programmes, where the number of holdings introducing innovation variated from 23 to 20 000 (these figures are based on 15% of the regions/Member States collected good quality data).

The qualitative data indicates that the rural development programmes have contributed to a medium extent to the introduction of innovative approaches.

One of the measures that provided some innovative approaches to beneficiaries in rural areas was M331 (training and information), implemented by 32 regions in 13 Member States, with a total expenditure of EUR 92.2 million. More than 700 000 training participants completed a training successfully. The total number of participants that successfully ended a training activity accounted for 5% of the total number of people employed in agriculture, food and forestry in 2013. The measure improved relevant skills, as well as networking and capacity building. Nevertheless, the measure had a very limited impact, as only 17% of all the ex post evaluation reports found that the measure contributed positively to the improvement of economic diversification, and in most cases to a limited extent (36%).

5.1.3.4.Rural development programme’s contribution to the creation of access to broadband internet

For the vitality of rural areas, broadband access is key. Coverage of fixed broadband remained at 97% in 2015, and about 6% lower in rural areas. Next Generation Access technologies 72 were available to only 53.7% of the EU-27 in 2012 and around 80% in 2017. 73  

In general, effective internet take-up lags behind broadband coverage. In 2012, 72.5% of all EU households had subscribed to a broadband connection. In the rural areas of the EU, this was only 64.3% of the households. These figures were slightly higher in 2017 (75% and 68% respectively). The gap between households having subscribed to broadband connection in rural areas and at national level was 10.3 pp in 2008, 8.2 pp in 2012 and 7 pp in 2017, showing that rural areas are nevertheless catching up.

The main measure that contributed towards broadband access in rural areas was M321 (basic services for the economy and rural population). The quantitative data (provided in 19% of the reports only), shows that the rural development programmes created new or improved access to broadband internet in 1.4 million households or businesses (15% of the total rural population). The average number of households or businesses with new or improved internet access was 81 315 per region, with the lowest region having only 7 that benefitted from these changes.

The qualitative data shows that the rural development programmes’ contribution to the creation of access to broadband internet was positive in 35% of all Member States/regions. These effects can be attributed mainly to regions that effectively activated the fibre optic infrastructure and enabled additional phone lines, thus laying the ground for subsequent access to internet by the residents. 15% of the reports, however, found that the rural development programmers had limited effects on improving access to broadband internet. This was primarily due to administrative problems with the implementation, which started late and therefore delayed concrete results. Finally, 37% of the reports established that the initiative did not apply to the specific Member State/region.

5.1.4.Axis 4. Building local capacity (LEADER) 

LEADER (Axis 4) is a horizontal axis thus affecting aspects of the other EAFRD axes. The evaluation support study concluded that the rural development programmes have contributed positively and to a medium extent to building local capacities for employment and diversification through LEADER. This conclusion also goes along the European Economic and Social Committee’s report, where LEADER was praised by all the interviewed stakeholders, as they found that it provided beneficial results for both the regions and the communities. Furthermore, it is also confirmed in the public consultation, with a majority positive recognition by all questioned stakeholder groups.

The measures under this axis, especially M411 (Competitiveness), M412 (Environment), and M413 (Diversification) were often jointly programmed and evaluated in several Member States/regions. As a result, they are reported on as M41 (Implementation of local development strategies). Some measures of the other Axis (mainly of Axis 3) were also programmed under LEADER. The table on next page gives an overview of the main objectives under each LEADER measure.

Table 2. Main objective of LEADER measures

Source: DG Agriculture and Rural Development

Member States spent a total of EUR 5.6 billion on LEADER measures, which were implemented as follows:

·Measure 411(competitiveness): 22 Member States; 66 regions; total EAFRD expenditure EUR 471.5 million.

·Measure 412: 16 Member States; 37 regions; EUR 58.9 million;

·Measure 413: 27 Member States; 88 regions; EUR 3.977 billion;

·Measure 421: 27 Member States; 74 regions; EUR 170.8 million;

·Measure 431: 27 Member States; 88 regions; EUR 897.7 million.

Figure 14. Expenditure per measure under axis 4

Note: M411 corresponds to competitiveness; M412, environment/land management; M413, quality of life/diversification; M421, implementing cooperation projects; M431, running the local action group, skills acquisition, animation

Source: DG Agriculture and Rural Development

As regards the quantitative overview, the output indicators reported for Measures 411, 412, 413 (together referred to as M41), are aggregated. Overall, measure 41 resulted in a total of 2 417 Local Action Groups (LAGs) covering 4.1 million km2 and 143 million inhabitants and 18.5 million beneficiaries. In total 222 794 projects were financed by the local action groups, i.e. on average 92 by LAG.

Measure 421 supported a total of 5 524 cooperation projects in which 7 432 LAGs cooperated with each other. 13 ex post evaluation reports provided information on the creation of in total of 4 784 jobs (gross value). Over a quarter (27%) of the evaluation reports found that measure 421 contributed to the LEADER approach to a great extent. 17% reported to a medium extent, 4% a limited contribution and 13% an unclear contribution.

Measure 431(running the local action group, skills acquisition, animation) supported 349 061 action groups and 5 863 participants ended a training activity successfully under this measure.

However, the output indicators have to be treated with caution as there are major issues regarding the land size and the population covered. E.g. according to the output indicators, the LAGs cover approximately 200% of the total rural area in Europe (not possible by definition) or 93% of the total land area.

5.1.4.1. LEADER’s contribution to building local capacities for employment and diversification

About half of the Member States reported on the contribution of LEADER to job creation. The figures are small, however, the quantitative information is not sufficiently consistent to allow for a judgment. According to the ex post evaluation reports, 28 242 jobs were created through M411 (competitiveness) in the 15 Member States that reported this indicator, 6 505 through M412 (environment/land management) in 3 reporting Member States and 12 525 through M413 (quality of life/diversification) in 13 reporting Member States. Additionally, 149 534 participants ended a training successfully through M411 (competitiveness), 58 through M412 (environment/land management) and 11 998 through M413 (quality of life/diversification).

According to the available qualitative information, measure 41 contributed to enhancing employment to a limited extent. 40% of the ex post evaluation reports found that M41 (M411, M412, M413 together) had a positive effect on employment. 15% reported a limited effect, 7% no contribution and 11% other effects. In addition for 27% no information was available.

The contribution of M431 (running the local action group, skills acquisition, animation) to build local capacities for employment was considered positive in 19 out of 82 reports (23%). 12% reported a limited contribution and in 42 reports (51%) no data was available. This leads to the conclusion that the measure contributed to building local capacities for employment to a medium extent, with 65% of all judgments being positive.

Only 10% of the evaluation reports reported positive effects of M41 (M411, M412, M413) on diversification. 13% reported a limited effect, 6% no contribution and 1% other effects. While the contributions to employment can be considered plausible, the evidence presented in the Member States’ evaluations on diversification was weak and thus not plausible.

5.1.4.2. The Local Action Group’s contribution to achieving the objectives of the local development strategies and those of the rural development programmes

The local development strategies are designed to reflect a multi-sectoral approach, where the priorities cover the needs and expectations of the stakeholders of the regions.

The contributions of M41 (aggregated M411, M412 and M413) to achieve the objectives of the total rural development programmer was deemed positive in 34% of the evaluation reports and with limited effect in 17% of the reports. LEADER contributed to the three main axes of the rural development programmer and to the crosscutting objectives of the EAFRD.

LAGs played an important role in preparing and implementing the local development strategies. The contribution of LEADER to achieve the objectives of the local strategies was considered positive in 36% of the reports. A number of reports stated a limited contribution (17%). However, as in a substantial number of reports no data was available (35%), the overall contribution is judged to be small.

The qualitative overview indicates that the LAGs have contributed to achieving the objectives of the local strategy and the rural development programmes to a medium extent, yet, due to the limited share of reports that provided conclusions, this qualitative assessment of the contribution has to be taken as indicative only.

5.1.4.2. LEADER’s contribution to improving local governance

EUR 897.7 million was spent on M431 (running the local action group, skills acquisition, animation) by 27 Member States across 88 regions, which resulted in 5 863 participants that successfully ended a training activity and a total of 349 061 actions supported. Unfortunately, it is not clear what kind of actions or projects were supported.

The contribution of M431 to local governance was considered “high” (30%) to “medium” (21%), although many data was missing in several evaluation reports (28%). One of the clearest effects of LEADER was the implementation of public/private partnerships, enabling stakeholders’ long-term involvement in the development of a local strategy. In general, this effect was hard to quantify given the lack of indicators to characterise local governance.

The qualitative overview indicates that the implementation of the LEADER approach contributed to improving local governance to a medium extent, however this assessment is based on a low level of input from Member States evaluations.

Conclusion

The overall assessment of LEADER cannot be considered plausible due to the small share of reports that provided conclusions (with the exception of the contribution to enhancing the employment situation). A general issue is that quantifiable indicators were not able to capture the specific characteristics and objectives of the LEADER approach, e.g. improving local governance or increasing local participation. The aspects that were quantified are not highly relevant for LEADER (e.g. job effects).

As Axis 4 is horizontal, it affects aspects of the other EAFRD axes and some measures under other axes, especially Axis 3, were programmed under LEADER. At times it was not clear whether such measures were reported on under Axis 3 or 4 in the evaluation reports. The synthesis approach did not sufficiently allow for consistently dealing with this aspect, amongst other specifics of LEADER.

Furthermore, consistency was weakened by evaluators using different approaches in answering the evaluation questions.

5.2.Efficiency

5.2.1.Qualitative assessment

The majority of ex post evaluation reports do not provide a judgment on efficiency of resource allocation at programmer level: 37% do not provide information or judgments at all and 23% only provide a judgment at measure, axes or objectives level. Out of all the reports that provide a judgment, 45% judge the resource allocation in relation to achieved outputs as “efficient”, while 13% report low levels of efficiency (inefficient), and 5% suggest no efficient allocation at all. Yet, only about half of the ex post evaluation reports (52%) provide a definition of the term efficiency and only 33% provide a definition that is clearly in line with the definition used in the Better Regulation Guidelines.

The public consultation assessment of efficiency is close to the one in rural development reports. Overall, 44% of the respondents thought that the rural development programmes were good or very good value for money, 31% thought they had limited value for money and 10% no value for money, with no major difference between stakeholders.

The difficulties to demonstrate the efficiency of the 2007-2013 Rural Development Policy were also amply highlighted in the Court of Auditors special report 12/2013 74 . The Court observed that the rural development programmes did not sufficiently set specific objectives with targets for the results and that the indicators were not sufficiently reliable. The European Court of Auditors also pointed to certain issues of efficiency. For instance, in the Special Report No 20 of 2015 75  they pointed out in relation to non-productive investments (measure 216) that “overall, the support has contributed to the achievement of objectives linked to the sustainable use of agricultural land, but in a way it was not cost-effective”. Similarly, in their Special Report No 12 of 2015 76 , related to the measures promoting a knowledge-based rural economy (measures 111, 114, 115 and 331) the Court found weaknesses related to cost-effectiveness. They also concluded that there was considerable scope for making real savings in Rural Development project grants in the 2014–2020 programming period by better approaches to controlling the costs. (Special report No 22 of 2014) 77 .

The European Economic and Social Committee (EESC) concluded in their report that there is a strong need for further simplification and less bureaucracy both at EU and Member State level, as well as at the regional and local level, when it comes to all procedures, from application through implementation to final reporting 78 . It stated that procedures were too complex and that this discouraged potential beneficiaries. The Committee also pointed to the strong criticism expressed by stakeholders to the long time needed to get the programme running.

Given the lack of robust evidence mentioned above, it is very hard to derive an overall conclusion on the proportionality of Rural Development expenditure with regard to the benefits achieved at EU level. It is clear however that there are wide variations in efficiency between programmes and between measures. Member States evaluations pointed to several factors limiting efficiency:

·The regulatory framework: the evaluations pointed to an, in general, “inappropriate regulatory framework and application of the law”. Fundamental design errors of the legal framework, individual regulations and very rigid legal interpretations in terms of error prevention were more pronounced. The changes resulting from the ‘health check’ reform (increasing demands on documentation, data and reporting and a growing number of controls and supervisory authorities), were also seen as limiting the efficiency.

·Design of programmes and measures: One report stated that 'standard measures', which are already offered over various funding periods with relatively large financial volumes and are more efficient due to the administrative routine and greater acceptance by beneficiaries than newer measures, which often also have lower financial budgets.

·Programme implementation: lack of staff and its insufficient availability, leading e.g. to long waiting time for beneficiaries, or the delays in starting the programmes are often mentioned. Another issue raised is the “lack of innovative actions”. For Axis 2 (environment and the countryside), evaluators suggest “increased actions for environmental development” and “improved knowledge structures, indicators, research, development work to strengthen the instruments that capture variables for environmentally friendly farming”. Improvement for axes 3 (improving the quality of life in rural areas) and 4 (Leader) could be the reduction of administrative burden, the development of qualitative indicators to highlight networks, social trust and belief in the future; better definition of objectives with regards to community led development.

·Low uptake of measures, low expenditure rates: this issue was found in several reports. The reduction of number of measures programmed with an increased focus on targeting could be a solution.

5.2.2.Quantitative assessment

Thus, since the Member States evaluations did not provide a clear answer to the efficiency question, an alternative approach, i.e. calculating the costs per result achieved, was used. This, however, also proved to be fairly difficult due to the low data quality on achievements, and it can only provide an approximation, which could not be used for a proper benchmarking.

The main limitation to the quantitative analysis of efficiency is that contextual conditions vary substantially between Member States. Therefore, there is no use in comparing calculated levels of efficiency between Member States. Furthermore, there are no benchmarks for the proportionality of costs available, neither from earlier evaluations of programmes, nor at the aggregate EU level. Where individual values for Member States are provided, they aim to illustrate the variability and they are not intended to judge overall proportionality of costs for this Member State.

A further limitation is that total EAFRD expenditure is reported per measure, but result indicators do not always match the measures. For example, result indicators are reported by operations for M111 (training), in relation to several measures (R4: M131 and M132) and in relation to some measures different indicators are reported (M121 to M124, M131, M311, M312). Expenditure cannot be disaggregated by operation or indicator, therefore efficiency is calculated for each indicator/ operation by dividing it by the total EAFRD expenditure. Thus the outcomes only provide a rough estimation of costs per results and do not take into account double counting.

Furthermore, the analysis made by the evaluator of efficiency did not include administrative costs.

5.2.2.1. Axis 1. Competitiveness

Cost per number of participants that successfully ended a training activity related to agriculture and/or forestry (RI1)

Achieved results for this result indicator were reported for only one measure M111 (training) for two criteria: 1) Passing with certificate, degree or diploma; and 2) Implementing the achieved skills. It is not obvious therefore if the achieved results could have been reported twice under both criteria. Of the 26 Member States which implemented M111, 13 Member States reported on both criteria and one did not provide any information. Extreme values are found in France, Ireland and Portugal, which can be caused by unspecified relation of expenditure to the two criteria reported for RI1. They were not considered in the calculation of average costs estimated around EUR 330/participant. Not all the expenditure in training (EUR 8.4 billion) lead participants to actually pass the training programs with a certificate and/or implementing its skills, thus the difference in expenditure.

Table 3. Participants successfully ending a training under RI1

Source: Evaluation support study

Note: the average cost exclude extreme values.

Cost for EUR 1 000 increase in GVA in supported holdings (RI2)

The calculations illustrate the costs for an increase in GVA by EUR 1 000. The attribution of the effect to the intervention is not as clear as with the other Result Indicators, as changes in GVA in supported holdings are influenced by a variety of factors, and not only by the measures. The results of this calculation have therefore to be understood as indicative only.

In total, 10 Axis 1 (competitiveness) measures are related to RI2; Denmark, Luxemburg, and the Netherlands did not report any data on this indicator. Some data provided were negative: to be interpreted as expenditure that resulted in a decrease in GVA. The average cost to achieve an increase of EUR 1 000 of GVA across all reporting Member States is EUR around 330. However, the variability is enormous, and 7 measures reported a negative relation between costs and GVA increase. Extreme values are found in Belgium, Latvia, and Spain, which may indicate inconsistency in unit of measurements for RI2. They were not included in the calculation.

Among the relevant measures, and taking into account the evidence base, M122 (improvement of the economic value of forests), and M123 (adding value to agricultural and forestry products) have shown the highest effectiveness in cost per achieved output ranging from EUR 100 to EUR 150 of EAFRD expenditure per EUR 1 000 achieved GVA in supported holdings. Furthermore, with an overall average one-time cost of EUR 333 per EUR 1 000 increase in GVA each year, it can be said that the measures under RI2 have efficiently contributed to increasing GVA in agriculture.

Table 4. Achieved increase in GVA under RI2

Source: Evaluation support study

Cost per number of enterprises introducing new products and/or techniques (RI3)

On the 4 measures that reported on RI3 (M121 – Modernisation, M122 – improvement of the economic value of forests, M123 – adding value to agricultural and forestry products, M124 – cooperation), a total of 73 valid entries have been made. The introduction of new products or techniques has cost on average close to EUR 80 000 per enterprise. Out of the 4 relevant measures, M124 (Cooperation for the development of new products, processes and technologies in the agriculture and food sector and in the forestry sector) shows the best efficiency with an average cost of EUR 17 800 per enterprise.

It has to be noted that there are large variations in the total number of enterprises reported under this result indicator, pointing at some methodological issues and leading to a wide range of estimated costs from EUR 2 550 to EUR 2 620 000 per enterprise.

Table 5. Number of enterprises introducing new products and/or techniques under RI3

Source: Evaluation support study

Cost per EUR 1 000 of agricultural production under recognised quality label achieved thanks to CAP support (RI4)

Values of agricultural production under recognised quality label were disaggregated between European and national labels/standards. Due to a low number of reporting Member States (15 for EU labels and 14 for national labels) the results have to be treated with caution. Several countries did not report any values despite implementing at least one of the relevant measures (Austria, Denmark, Estonia, Germany, Latvia and the Netherlands). The average costs calculated per achieved EUR 1 000 of production value vary between EUR 8 100 for EU labels and EUR 12 500 for national labels. The range between minimum and maximum cost also varied significantly, from EUR 2.4 to EUR 2 019.5 for EU labels and from EUR 0.9 to EUR 62 660 for national labels.

Cost per number of farms entering the market (RI5)

Only 12 Member States implemented at least one of the two related measures, and only four of them reported on RI5, namely Bulgaria, Cyprus, Lithuania and Romania. Thus, the relevance of the findings about efficiency at an aggregate level is low. The calculation of cost per achieved results hints on an average cost for a farm entering the market of EUR 108 770, with a minimum cost of EUR 10 280 and maximum of EUR 3 million.

5.2.2.2. Axis 2. Environment and the countryside

Cost per ha under land management (RI6)

Only one result indicator (RI6) is applied to monitor outcomes of Axis 2 (environment and the countryside) measures. Still, RI6 “Areas under land management (ha)” reports separately for the different objectives biodiversity, water quality, climate change, soil quality and avoidance of marginalisation. A slightly different approach is taken here by calculating the cost at the level of measures, as the same hectares (in which the corresponding RI6 is expressed) can serve several purposes. Particularly for water quality, climate change, soil quality less results were reported. Examples for biodiversity and avoidance of marginalisation are provided in the table below. As there are too many gaps in data on the other objectives; it is not possible to know which part of the expenditure was used for actions targeting specific objectives; and it is not possible to rule out double counting, a calculation of cost per benefits for those would be not appropriate.

Table 6. Average cost per supported ha under land management (RI6)

Note: For the full label of measures see Table 1 . Measures with 5 of less Member States reporting were dropped.

Source: Evaluation support study

There are large variations in the costs per supported ha, which can be to some extent attributed to the limitations in the calculation but also to the level of ambition of the commitment. The most cost efficient measures across both objectives appear to be M213 (Natura 2000), M225 (Forest-environment) and M226 (Restoring forestry).

Some of the forestry measures (particularly M221 to M223, related to first afforestation of agricultural and non-agricultural land) are less cost efficient than those related to agricultural area for both of the considered sub-indicators biodiversity and avoidance of marginalisation. However, this can be due to the cost intensive nature of these measures.

5.2.2.3. Axis 3. Diversification of the rural economy and quality of life in rural areas

Cost for EUR 1 000 increase in non-agriculture GVA (RI7)

Measures related to RI7 were implemented in 25 Member States out of which five did not report any data: Belgium, Cyprus, Denmark, Luxemburg and the Netherlands. The calculations show how much it costs to increase non-agriculture GVA. On average this amounts to a one-time cost of EUR 2 000 for an increase in non-agriculture GVA of EUR 1 000 per year. Again, variations are significant between Member States. However, there seems to be no significant difference in efficiency between the different measures. Extreme values are found in Latvia, which may indicate inconsistency in unit of measurements for RI6. They are not included in the calculation of minimum, maximum and average costs.

Table 7. RI7 increase in non-agricultural GVA

Source: Evaluation support study

Cost per gross number of jobs created (RI8)

This indicator is relevant not only for Axis 3 (diversification) measures, but also for most Axis 4 (Leader) measures. However, no data was reported on LEADER-related measures in the annual monitoring tables. Axis 3 measures reported under RI8 were implemented in 25 Member States. The calculations result in an average cost per job created of around EUR 29 000. However, there is a large variability between measures and between Member States.

M312 (Support for business creation and development of micro-enterprises) has the highest efficiency with about EUR 22 500 of EAFRD expenditure per job created; however this calculation does not take into account the provided national or private expenditure that relates to the measure.

Table 8. Gross number of jobs created under RI8

Source: Evaluation support study

Cost per additional number of tourist visits (RI9)

Only M313 (Encouragement of tourism activities) is reported under RI9, which was implemented in 22 Member States. RI9 was reported by two separate measurements: “number of overnight stays” and “number of day visits”. As there is no information whether a single activity could have led to effects in both categories, and no disaggregated expenditure data available, the sum of both indicators is used in the calculation. The average cost per additional visit was calculated to be EUR 35.

Table 9. Additional number of tourist visits under RI9

Source: Evaluation support study

Cost per addition person in rural areas benefiting from improved services (RI10)

The three related measures for RI10 have been implemented in 26 Member States. The average cost for an additional person benefitting from improved services is EUR 52. An extreme value is found for one Member State (Malta), which might indicate an inconsistency of units of measurement. It is not included in the calculation of average costs.

The calculations made by the contractor in the evaluation support study show that M323 (Conservation and upgrading of the rural heritage) has the lowest average EAFRD expenditure per person benefitting from improved services with only EUR 17 per person. However, the large difference in relation to the other relevant measures can possibly be explained with their different scopes.

Table 10. Population benefitting from improved services

Source: Evaluation support study

Costs per person benefitting from increase in internet penetration in rural areas (RI11)

The increase in internet penetration in rural areas is measured through the population that has access to (broadband) internet connection. The indicator is only relevant for M321 (basic services for the economy and rural population). Extreme values were found in Greece, the Netherlands and Austria, which may indicate inconsistency in unit of measurements for RI11.

Table 11. Population benefitting from improved internet connection

Source: Evaluation support study

5.2.2.4. Axis 4. LEADER

Cost per participant that successfully ended a training activity (RI12)

In 17 Member States, measures reported under RI12 have been implemented, and Member States all reported on the indicators (though not all for each of the measures). The average cost per participant that successfully ended a training activity was EUR 200. An extreme value is found for one Member State (France), which might indicate an inconsistency of units of measurement. It is not included in the calculation of average costs.

Table 12. Average cost per participant successfully ending a training activity

Source: Evaluation support study

5.3.Coherence

5.3.1. Coherence of rural development policy with other CAP funds and other EU interventions in the same programme area

   The assessment of the coherence and complementarity of the rural development programmes with other CAP funds is restricted by the fact that only five programmes reported on it 79 . Most Member States did not address this issue in their ex post evaluations, since it was only part of the ex-ante assessments of the rural development programmes, and not mandatory for the ex post evaluation. Furthermore, the information available is limited to such an extent that only a qualitative description of the five cases is available.

The ex-ante assessment had already concluded that generally the programmes did not always look actively for synergies between the different EU interventions in the same programming area, but rather for clear demarcation and avoidance of overlaps between different funds 80 . The other significant funds in rural areas included the EU Structural and Investment Funds (ESIF), which include the European Regional Development Fund, focusing on regional development, economic competitiveness and infrastructure; the European Social Fund, focusing on the development of human capital, employment and social inclusion and the European Agricultural Fund for Rural Development.

The ex post evaluations showed that some Member States strongly differentiated and ensured no overlapping between the funds. Nevertheless, this approach also had the negative effect of lack of synergies. In most cases, ensuring complementarity between these funds took the form of establishing demarcation lines and coordination mechanisms. Only in some cases a more strategic vision for the coordinated use of EU funds was chosen by Member States. The guiding principles were set in the National Strategy Plans, while some programmes provided a detailed description. The coordinated use of EU funds was ensured also by the different managing authorities during the start-up phase of the programmes For example, in the programmer of Brandenburg/Berlin a range of complex projects were identified and coordinated in a coherent manner, integrating funding possibilities towards a single goal. The ex post evaluations of Castilla La Mancha and Latvia also attribute the successes to clear coordination.

Communication and coordination were deemed important for complementarity and synergies between Community instruments, yet issues were found in the coordination in two programmes (England and Hamburg). Furthermore, the EESC in its report 81 pointed to a lack of coordination, complementarity and consistency with the actions of the other structural and cohesion funds that hindered the overall aim of a more balanced territorial development.

These rather mixed (with very small number of programmes reporting) results are in contrast with the results of the Public Consultation, where 68% of the respondents found external coherence positive (sum of very positive and moderately positive view) and only 12% found a lack of it, with stakeholders “farms” and “individual citizens of rural areas” finding it less consistent than the rest of the groups (but still mostly positive). The other 20% had no opinion.

5.3.2. Coherence with the overall rural development priorities

The outcomes of the rural development programmes are to a varying extent consistent with the four Rural Development objectives. While relatively few instances of issues being reported can on the one hand be interpreted as an absence of contradictory effects, this might also be due to evaluators having difficulty in grasping such effects. The internal coherence with regards of the specific objectives is the following

·Objective 1 (improving the competitiveness of the agricultural and forestry sector), was achieved to a moderate extent. a small share of ex post evaluation reports stated negative and contradictory effects on competitiveness, bargaining power and labour productivity/profits, though the overall ex post analysis showed a moderate positive coherence. A negative contribution in this respect was reported for only one programme and for one measure (M142 – setting-up producer groups). The negative effect on competitiveness was based on negative effect on GVA, productivity, and employment. Contradictory other effects were reported in some cases from M141 (Semi-subsistence farming) with relation to the decreased bargaining power of non- beneficiaries; for M224 (Natura 2000) payments were considered partly as insufficient (taking into account the long production period of a forest (> 100 years); as for M311 (diversification) the negative effect was linked to labour productivity/profit; and for M312 (business creation) the negative effect was due to decreased competitiveness.

·Objective 2 (improving the environment and the countryside) was considered overall positive, however the extent of coherence couldn’t be quantified for all domains. Considering the individual domains the extent ranges between moderate and high, as far as quantifiable: climate change mitigation: high extent, water management: high extent, protection of natural resources/landscape: moderate extent, renewable energy supply: positive impact, but difficult to quantify.

For a small share of programmes, negative environmental effects were reported, which are arising from M121 – modernisation (while at the same time, the overall environmental effect of the measure was stated to be positive), M132 – Quality schemes. The nature of these effects was not further specified.

·For objective 3 (improving the quality of life in rural areas and encouraging diversification of the rural economy), although there were no negative contributions nor contradictory other effects identified in the ex post evaluation reports and positive contribution was stated in the few ex post evaluation reports that reported on it, it is difficult to determine and quantify the overall coherence due to the lack of a higher-quality evidence base.

·For objective 4 (building local capacity for employment and diversification), there are no negative contributions or contradictory other effects at measure level, however the coherence of the programmes with objective 4 is difficult to qualify.

Overall, the coherence with the rural development objectives tend to depend on the quality of the evidence base reported, which is higher for the longer-established domains of Axis I and II. This results in higher visibility of these axes’ contributions, as compared to more complex domains such as quality of life and newer approaches such as LEADER.

5.4.Relevance

The analysis of the relevance of the Rural Development measures is based on the needs listed beneath, covering the social, economic and environmental dimensions. These needs are then compared to the programmes’ contributions addressing them.

Table 13. List of needs being addressed by the rural development measures

Source: Evaluation support study.

Furthermore, the magnitude of the expenditure by axes correspond to the main needs, thus expenditure shares can be used as a basis of the analysis of the needs, enabling an overview at EU level. Nevertheless, it is not feasible to relate regional-level needs to the way they were addressed in the same region. It is only possible to provide a comparison of the main EU-level needs to how they were addressed by programmes across regions overall. Moreover, these needs identified at EU level in the synthesis of ex-ante evaluations are not specified in detail, so only a part of them can be linked precisely to topics covered by Rural Development measures and related indicators.

Regarding the need of “structural adjustments and modernisation”, despite not having a specific result indicator associated (but part of the Axis I needs), it was considered in the reports as positively addressed.

Similarly, for need “value chains, added value, integration between sectors” the increase in gross value added in supported holdings/enterprises (RI2) demonstrates the relevance of the policy.

For “lack of specialisation/diversification/de-concentration/quality”, in the Member States/regions reporting on it, the increase in the value of agricultural production under recognised quality label/standards highlights the policy relevance. However, due to a very low number of Member States reporting on the indicator, these results cannot be extrapolated for the whole EU-28.

In the social/socio-economic dimension, overall the rural development programmes addressed the issues of the relevant social needs to a moderate extent. The rural development programmes helped to create new job opportunities, which amounted to respectively 1.2% and 0.4% of the total number of people working in agriculture and forestry.

In addition, through the rural development programmes’ interventions, 154 million people could benefit from improved services in rural areas and around 15% of the total rural population could benefit from new or improved internet services in the rural areas (RI11).

The last two needs, “Demographic change” and “Physical infrastructures” do not have a relevant matching result indicator, and therefore its relevance can only be addressed through the reports. Physical infrastructures was judged by Member States/regions as positively addressed (yet only moderately). For the demographic change need, the results were not so clear, and depended on the specific domain the measures were acting on.

As regards the environmental dimension, overall, the rural development programmes addressed the issues of the relevant environmental needs to a moderate extent. With a volume of expenditures that amounts to the highest share among the three dimensions (i.e. nearly twice the average).

More than half of the ex post evaluations concluded that the rural development programmes had a major positive contribution to protect and enhance natural resources and landscape, including biodiversity and high nature value farming and forestry. Half of the ex post evaluations also found that the rural development programmes played a very positive role in improving water management.

5.5.EU Added Value

EU added value was not addressed in the ex post evaluations carried out by Member States, and thus even the qualitative information for the judgment is very limited, therefore the analysis is based on the results of the effectiveness and efficiency analysis, the assessment of coherence, and conclusions on subsidiarity from the synthesis of the ex-ante evaluations. The EAFRD funding via the rural development programmes ensured EU added value to a certain extent, and it depended on the specific aspects considered. Nevertheless, the quality of reflection upon the question of EU added value is not sufficient in the ex post evaluation reports nor there is information derived from the relevant assessment results. In addition, there is no common approach to assess subsidiarity and proportionality at programme level. Where conformity with these criteria is declared, it is not underpinned with descriptions of the mechanisms ensuring them.

The evaluation can conclude that the implementation of EAFRD funding via rural development programme has been effective in achieving EU level objectives and supporting key EU priorities to a variable extent. It appears that programmes have been more successful and effective in achieving Axis I and II objectives. However, the lower extent reported to achieve Axis III objectives could also be a result of overall difficulties in measuring the effects.

Regarding the principles of subsidiarity and proportionality, there are good examples for the various CAP supports’ EU added value, e.g.: private and civic actors are involved at all levels of decision making in the programming process (Czech case study); the financing decisions can be taken more closely to the final beneficiaries. The EU added value of funding is especially relevant, where support of rural areas would otherwise be very difficult (Greece case).

The EU added value is illustrious in the case of LEADER. For example, in the Brandenburg case study it was found that the LEADER method not only illustrates the successful capacity building of subsidiary planning and decision-making structures, but it is also in line with the objectives of an overarching federal structure. The state had very little role in the identification and selection of projects and was thus only able to bring federal cultural policy goals into play to a limited extent. The projects were mainly aimed at intra-local and micro-regional use within the respective LEADER area. The micro-regional orientation of the projects is a result of bottom-up planning.

With regards to the public consultation results, respondents were majorly positive about the EU added value and only few respondents did not see the benefits of EU funding instead of Member State action, or had a feeling that the EU funding leads to a distortion of competition. In general, EU funding is considered a stable funding source, which also led to an increase in national funding through the co-financing principle. EU funding can also have a spill over effect and attract other developments in rural areas. Respondents indicated that it is important to them that EU funding is not influenced by local political dynamics and that there is more equality between the beneficiaries. Furthermore, the variety between projects that could be subsidised with EU funding was perceived higher than national funding.

The positive perception of EU added value was also confirmed by the European Economic and Social Committee in their report 82 , particularly in relation to the early and effective inclusion of civil society organisations in the bottom-up preparation and implementation of the programmer, which is emblematic in the case study on Tuscany. However, there are cases where the added value was rather poor, e.g. in Poland, where small NGOs were given little room to try and influence the decisions taken by the government.

6.Conclusions

The ex post evaluations for rural development programmes, strategies and frameworks for the period 2007-2013, as well as the stakeholders consulted are rather positive about the contribution of rural development programmes, however the extent of achievement of the four objectives differs.

A majority (59%) of the Member States/regions stated in their ex post evaluation reports that the effects of the CAP on competitiveness were positive (25% reported limited impact and 16% negative). The support study thus concluded that overall the competitiveness of the agricultural and forestry sector (objective 1) improved to a moderate extent. EU labour productivity increased by 4.1% thanks to the rural development policy. Investments aiming to improve productivity, efficiency and sustainability were key to increase GVA in the primary sector while also creating positive spill-over effects on the whole economy, namely investments in farm modernisation (M121), adding value to agricultural and forestry products (M123), and improving infrastructure in rural areas related to the development and adaptation of agriculture (M125), training and information actions (M111).

The evaluation support study found that the generational-renewal measures (early retirement (M113), setting-up (M112) and investment support) did not influence significantly the number of young farmers across the EU as a whole. They rather targeted the maintenance and protection of farming jobs, particularly in marginal areas, rather than the creation of new farming jobs.

The support study could not conclude on the contribution of the rural development policy to improved competitiveness in the overall rural economy, due to the limited number of reports.

As regards the environment and countryside (objective 2), although the status of natural resources and particularly biodiversity deteriorated over the programming period, Member States/regions reported positive contributions from the rural development policy, suggesting that the deterioration would have been worse and land abandonment would have been higher without the CAP. On this basis, the evaluation support study concluded that the rural development programmes’ interventions contributed to the protection of natural resources and landscape (to a moderate extent) and climate change mitigation (to a low or unclear extent). 61% of the ex post evaluations reported positive effects on the protection and enhancement of natural resources and landscape, particularly biodiversity and High Nature Value farming and forestry. The agri-environmental measures (M214) were of particular importance with respect to protecting natural resources and biodiversity also due to the fact that all programmes implemented the measure and it accounted for the largest proportion of expenditure (26%).

Measures with the core objective of preventing land abandonment, (M211, M212) had a positive effect particularly on biodiversity and high nature value farming/forestry, mainly through maintaining extensive agricultural systems in areas where appropriate land management is important for biodiversity and High Nature Value habitats. They also helped retain population and employment in fragile rural areas.

Renewable energy production was not the specific focus of any rural development measures. The qualitative data showed that the rural development programmes positively contributed to the supply of renewable energy, but only to a very limited extent (in 12% of the Member States/regions), which was based on the support for biogas and biomass energy. In some regions, investments in solar energy and distribution systems contributed to an expansion of renewable energies. Renewable energy production was reported on it in relation to measure M114 (use of advisory services by farmers and forest holders) and M311 (diversification to non-agricultural activities). The ex post evaluation reports indicated that Member States or regions enhanced the transition from fossil sources to renewable biomass based sources, installed renewable energy production plants or generally produced energy from renewable sources through M311.

As regards climate change mitigation and adaptation, overall only 6% of the Member States/regions reported on the environmental measures’ positive effects in relation to mitigation, which were however seen rather as a side effect than as a result of clear targeting in the rural development programmes.

The ex post evaluations very rarely reported on water management issues, therefore the effects of the CAP on water use and water quantity could only be assessed qualitatively and with difficulty. Nevertheless, overall more than half of the ex post evaluation reports indicated that the rural development programmes positively contributed to water management.

Out of the four objectives the rural development programmes’ contributed to improving the quality of life in rural areas and encouraging diversification of the rural economy (objective 3) to the most limited extent. However there are fewer measures with a direct effect on this objective and the lowest evidence and low quality data in Member States/regions evaluation reports. In addition, interventions encouraging diversification of the rural economy prove to produce less direct and measurable effects in the short term. The CAP’s impact on quality of life can be mainly attributed to basic infrastructure services (e.g. mobility and communication) and the social dimension (such as governance, social sustainability).

The quantitative data shows that the rural development programmes contributed to create new jobs, not just in the primary sector but also in the entire economy, particularly food and tourism, but also culture and services. The creation of employment prospects, especially for young generations in rural areas also limited the abandonment of farms, notably in mountainous regions.

The recent JRC study on the effects of the CAP proved the positive effect of the CAP in general and of rural development policy in particular with regard to not only safeguarding employment in agriculture, but also creating jobs in rural areas as well as boosting the gross value added.

The European Economic and Social Committee showed in its report that rural development programmes contributed to maintain agriculture and promoted competitiveness of rural businesses and employment and diversified rural economies.

While rural development programmes have been overall successful in promoting competitiveness, results under innovation do not seem to be very significant. The introduction of innovative approaches as a result of the policy was rather small, and mainly covered beneficiaries in rural areas through M331 (training and information). The main measure that contributed towards broadband access in rural areas was M321 (basic services for the economy and rural population). The quantitative data (provided in 19% of the reports only), shows that the rural development programmes created new or improved access to broadband internet in 1.4 million households or businesses (15% of the total rural population). The average number of households or businesses with new or improved internet access was 81 315 per region, with the lowest region having only 7 that benefitted from these changes.

Regarding building local capacity for employment and diversification (objective 4), the evaluation support study found that the local action groups contributed to a limited extent to achieving the objectives of the local strategy and that of the rural development programmes, however the rural development programmes have contributed to a medium extent to building local capacities for employment and diversification through LEADER.

However, the overall assessment of LEADER cannot be considered plausible due to the small share of reports that provided conclusions (with the exception of the contribution to enhancing employment). In addition, the quantifiable indicators were not able to capture the specific characteristics and objectives of the LEADER approach, e.g. improving local governance or increasing local participation. Furthermore, as Axis 4 is horizontal, it affects aspects of the other EAFRD axes and some measures under other axes, especially Axis 3, were programmed under LEADER. At times it was not clear whether such measures were reported on under Axis 3 or 4 in the evaluation reports. The synthesis approach did not sufficiently allow for consistently dealing with this aspect, amongst other specifics of LEADER.

As regards the efficiency of the rural developments, it is difficult to assess the proportionality of costs to the benefits achieved based on the ex post evaluation reports and the available input, output, and result indicators tables. Overall, 62% of reports provided some sort of judgment regarding the efficiency of resources allocated to the programmes. Taking into consideration the limitations of the data, it was not possible to provide an overall judgment of proportionality of EAFRD expenditure with regards to benefits achieved at the EU level. The difficulties to demonstrate the efficiency of the 2007-2013 Rural Development Policy were also highlighted in the Court of Auditors special report 12/2013 83 . The Court observed that the rural development programmes did not sufficiently set specific objectives with targets for the results and that the indicators were not sufficiently reliable.

The evaluation support study also concluded that the limitations were also attributed to the regulatory framework, particularly the very rigid legal interpretations in terms of error prevention. In addition, the changes resulting from the policy revision called the ‘health check’ leading to high adjustment costs due to increasing demands on documentation, data (including IT systems) and reporting, as well as a growing number of higher-level controls and supervisory authorities, were also seen as limiting the efficiency. Some reports also raised shortcomings with regard to the steering structure of the programmes and measures, including the lack of staff, and the low expenditure rates on some of the measures. Often the causes and effects of these issues are interrelated.

Considering the coherence of the programmes with the four rural development objectives/priorities, it was judged to a limited, or moderate extent. Negative contributions to the objectives were marginally reported. The consistency of rural development programme projects with other funding from the first pillar of the CAP has been evaluated only in the ex-ante assessments, therefore, a conclusion cannot be provided based on the information available.

As regards the coherence and complementarity with other CAP funds, the rural development programmes aimed at clear demarcation and avoidance of overlaps between the funds. However, this approach had the negative effect of lack of synergies.

Overall, the rural development programs have contributed to addressing the needs in the programme area to a moderate extent. In the social/socio-economic and economic dimensions, there are considerable differences in the extent to which individual needs were addressed by the rural development programmes, while in the environmental dimension needs were addressed more homogeneously. More specifically, within the social needs, the rural development programmes were moderately relevant (in terms of contribution) within the area of basic services and physical infrastructures and least for demographic change. Regarding economic needs, the programmes were moderately relevant in the area of value chains, added value, and integration between sectors. As for the environmental needs, the rural development programmes were moderately relevant as regards natural resources, sustainable practices and biodiversity, ecological structures, habitats, whereas the positive effects on climate change mitigation could be considered rather as side-effects of the positive impacts of the environmental measures and not the result of clear targeting.

The question of EU added value was not sufficiently addressed in the ex post evaluation reports. Based on the overall judgments on the three criteria (effectiveness in achieving objectives, coherence with EU priorities and complementarity with other instruments, and subsidiarity), it can be concluded that EAFRD funding via the rural development programmes ensured EU added value to a medium and variable extent. It needs to be considered that the judgment varies with regard to the individual criteria.

The conclusions of this evaluation is very relevant for the long-term vision for the EU’s rural areas 84 , which aims to enable rural areas to make the most of their potential and support them in facing their own unique set of issues, from demographic change to connectivity, the risk of poverty 85 and limited access to services.

The core lessons learnt relate to the monitoring and evaluation framework. The ex post evaluations on the 2007-2013 funding period were the first ones completed using a Common Monitoring and Evaluation Framework (CMEF) 86 , and adjustments to the framework were made even during the programming period. In particular, following the experience of the mid-term evaluations, the evaluation questions were reviewed and rationalised prior to the ex post evaluation exercise. Nevertheless, there is a wide variety in approaches and overall quality of the Member States evaluations. Thus, whilst the CMEF represents a step forward from the previous situation, the limitations in the relevance, calculation and use of indicators, and difficulties to establish and maintain realistic target make it difficult to fully demonstrate the outcome of the policy. The evaluation points to a clear need for an improved, better regulated data framework, including clear description of concepts, metadata and realistic targets. Any changes in targets need to be made explicit and related to the programme strategy, rather than shifts of resources between measures, and should be accompanied by a record of the timing and reason of the changes.

As it takes time for the impact of a policy to become visible, evaluation requirements and design should take into account the variable time lag between interventions expressed in achieved output and results, and the attributable impacts.

As far as the reporting structures are concerned, there is clearly a need for a more prescriptive comprehensive structure and maximum report size. It is also suggested to propose appropriate approaches and structures for those evaluation questions and information needs that are really relevant for EU-level policy information. The CMEF was reviewed and fully revised for the 2014-2020 period and took account of the lessons learned during the implementation of the 2007-2013 programmes 87 .

The results from the evaluation support study were available to feed into the preparation of the CAP post 2020 proposals. The introduction of the performance framework as laid down in the proposed CAP strategic plan regulation and the definition of the evaluation elements addresses issues raised in this ex post evaluation. This includes the definition of a clear indicator framework with agreed approaches to calculate the targets and indicator values, as well as clear and well-defined performance reporting obligations feeding into a performance review and a simplified and more flexible evaluation framework for Member States.

Under the new CAP, climate features prominently among the new CAP’s 10 specific objectives. In terms of policy design, each Member State must assess its needs and integrate all tools to address them in a single strategic plan. The new CAP largely discontinues the one-size-fits-all approach of direct support. Member States have more flexibility to design and combine the policy tools. Reducing the administrative burden and simplifying procedures is in the focus to improve outreach and uptake too. Increasing the attractiveness of rural areas requires integrating this policy with national policies. In particular, the policy for generational renewal in agriculture must be based on such an integrated approach, aided by substantial CAP funding.

Annex 1. Lead DG, Decide Planning/CWP references

Lead DG: Directorate-General Agriculture and Rural Development (DG AGRI)

Decide planning reference: PLAN/2016/527 - AGRI

1.Organisation and timing

This was a policy evaluation project included in the DG AGRI evaluation plan 2016-2020. It followed the Better Regulation guidelines (2017) with regard to evaluations. The evaluation work was carried out through an external evaluation study, contracted through a service request under a framework contract, conducted in conformity with the DG AGRI procedure for the organisation and management of policy evaluations carried out by external contractors. The project was supervised under the technical as well as the contractual management of AGRI unit C.4 in charge of Monitoring and Evaluation (As of 2022 unit A.3 “Policy Performance”).

An Inter-service Steering Group (ISG) was set up by the Commission in March 2017, with the mandate to provide information, prepare the terms of reference, monitor the work of the external study team, discuss and give advice on the approval of the final report, comment on the draft evaluation SWD.

The ISG was composed of the Secretariat-General of the Commission and DGs ENV, CLIMA, JRC, REGIO, RTD, MARE and AGRI (14 different units). The Steering Group started its meetings in March 2017 and held 6 meetings.

The evaluation roadmap was published on the 19th of June 2017 and set out the context, scope and aim of the exercise. The roadmap presented the questions to be addressed under the five categories of effectiveness, efficiency, relevance, coherence and EU added value. During the feedback period on the roadmap two contributions were received. None of them required changes of the approach towards the evaluation.

The evaluation project carried out by the external contractor started in August 2017. The final deliverable was received on 24th of April 2018 and accepted. The external evaluation study provided the basis for this SWD.

The EESC finalised on the 6th of October 2017 the information report “Ex post evaluation of rural development programmes 2007-2013” 88 .

2.Consultation of the RSB 

The RSB scrutinized this evaluation SWD in the meeting held on 3 July 2019 and provided a negative opinion. The comments raised have been addressed in the following way:

3.Evidence used

This evaluation was designed as a synthesis of the evaluations carried out by the Member States. The support study used the following inputs:

-91 Rural development programme ex post evaluation reports;

-27 National Strategy Plans;

-88 Rural development programmes;

-2 National Frameworks;

-National Rural Development Network programmes.

The Member States submitted this material to the Commission in the national language. The evaluation support study provides more details on the exact title and content of these reports. The ex post evaluation reports were drafted by independent country expert evaluators according to predefined templates.

Besides this, the Commission made available to the evaluator responsible for the support study the financial tables including expenditure per measure and axis for all rural development programmes (Input indicators), tables including the results for output indicators for each Member State and tables including the results for result indicators for each Member State.

Additionally, the evaluation could draw on input from the public consultation (see Annex 2) and the evaluation report NAT/699-EESC-2017-00690-00-00-RI-TRA carried out by the European Economic and Social Committee:

- https://www.eesc.europa.eu/fr/sections-other-bodies/sections-commission/agriculture-rural-development-and-environment-nat/information-reports ;

Also the synthesis of the ex-ante and mid-term evaluations of the rural development programmes 2007-13 were used:

- https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/cmef/rural-areas/synthesis-ex-ante-evaluations-rural-development-programmes-2007-13_en (synthesis);

- https://op.europa.eu/en/publication-detail/-/publication/aaefb39b-aa88-4f6f-b4dd-4bbd48c43c56 (mid-term).

4.Quality and limitations of the evidence used

In all the cases, and in spite of the adjusted methodologies, the quality and availability of data continued to be a limitation throughout the project.

1) In general, the quality of the rural development programmer ex post evaluation reports was not uniform. Some of the reports provided substantial analyses with well-argued examples, while other reports lacked clear reasoning and presented conclusions without substantiation. In addition, some reported values seemed out of the scope of possibilities, either being much larger or smaller than what would seem feasible for the specific aspect being measured.

2) A second issue is the mostly qualitative nature of analyses in the ex post evaluation reports, and the limited quantitative support to justify the effects claimed. Attempts to use output, result and impact indicators to validate or test the findings of the qualitative analysis were only to a limited extent successful due to the low quality of the data.

3) There were limitations to the data for indicators and targets: There are inaccuracies in the data from rural development programmer annual reports reported by Member States, as in the case of the data provided on the result indicators, and potential inconsistencies between data from rural development programmer annual reports reported by Member States and indicator values reported and used for the judgments in the ex post evaluation reports.

4) The output and result indicator targets have in many cases proven not to be realistic. This can be explained by the lack of experience of the Managing Authorities in setting targets. As a result, targets were often adjusted during programme implementation and were therefore not considered reliable to provide benchmarking. While adjustments throughout a life-cycle of a programme can reflect changes made to the programme, the adjusted targets were often drastically reduced and, when compared to the final outputs, not always substantiated. Without being able to use the targets, it was often difficult to assess the extent of the rural development programmes’ achievements. Where possible, result and output indicators were compared to the context indicators, and triangulation between the qualitative and limited quantitative data was used, but such comparisons could solve the problem to only a limited extent.

Annex 2. Stakeholder Consultation

Before contracting the support study, the Commission offered stakeholders the possibility to give feedback on the evaluation roadmap during the four week period from 20 June 2017 and 18 July 2017. Only two feedbacks were received, both related to rural development and agricultural policy in general and hence not requiring changes to the evaluation design.

In the framework of the 2007-2013 ex post evaluation report of each rural development programmer, there were already extensive consultations targeting the following main stakeholders: public authorities responsible for implementation of the rural development programmes including paying agencies and bodies delivering farm advisory services; farmers and farmers' organisations; academia and experts as well as NGOs and other civil society organisations active in the field of rural development. The aim of those consultations, which took the form of surveys and interviews, was to seek information and feedback from the relevant stakeholders and wider public in relation to practical experience with the implementation and the effects of each individual rural development programmer.

Also the European Economic and Social Committee (EESC) produced an information report 89 assessing three rural development programmer axes (Competitiveness; Environment and land management; Quality of life in rural areas and diversification of the rural economy), collecting data through: 1) online questionnaire to civil society organisations; 2) fact-finding missions (and analysis of case-studies) from Finland, Poland, (Andalucía) and Italy. Both questionnaire and fact-finding missions took place in the period April-May 2017.

In order not to duplicate work already carried out at Member State level, consultation activities consisted of the open-based public consultation and the cooperation with the (EESC).

For this public consultation, citizens were asked to give their opinion about policy that has undergone some changes during the 2013-2018 period. Rural development remains an ongoing process that is subject to innovation and policy change. Although it can be expected that a large part of the respondents have filled in the survey in the light of the post 2020 CAP reform, the results still give a good insight in the perception of the 2007-2013 policy among citizens and it can form a valuable input into the discussion on the CAP reform.

The internet based public consultation was held in all official EU languages from 22 January until the 23 April 2018. The survey results were analysed using the standard spreadsheet tools for the numerical and replies, and using the Commission’s DORIS tool for textual entries. A check for campaigns did not identify significant issues, as there were only three respondents with very similar replies.

All citizens and organisations were welcome to contribute to this consultation. In particular, contributions from the following rural development programmes beneficiaries were sought for:

·Farmers;

·Processors;

·Retailers;

·Wholesalers;

·Individual citizens of rural areas;

·National public authorities;

·Regional public authorities;

·Business organisations (including associations, chambers of commerce, etc.);

·Non-governmental and civil society organisations;

·Other public institutions.

At the time of the deadline, 720 replies were received, covering responses from all but two Member States (Malta, Cyprus). Of the respondents, 56% identified themselves as citizens, 44% as professionals or organisations.

Figure 15. Number of respondents per Member State

Source: Evaluation support study

Note: No responses from Cyprus nor Malta

In addition to the responses to the survey questions, 144 respondents gave additional comments on the rural development programme 2007-2013 and 26 unique documents were uploaded. Two documents were received outside the formal consultation context.

The format of the questionnaire followed the better regulation criteria: effectiveness, efficiency, relevance, coherence and EU added value. Most of the criteria were dealt with in closed questions, only for EU added value an open question was used. Besides the better regulation criteria, the questions focussed on the main goals of the rural development programmes. The following topics were addressed:

·Increasing the competitiveness of agriculture and forestry;

·Improving the environment, countryside and quality of life in rural areas;

·Encouraging diversification of the local economy and the local capacity to diversify;

·Building the local capacity for employment.

In general, a majority of the respondents gave a positive answer to the questions. No striking differences were found between the different stakeholder groups and between the beneficiaries of the different axis. However, some farmers are less positive about the impact and relevance of the rural development programmes. When assessing the results per stakeholder group in this report, the nuances between the groups are mentioned. Overall, the effectiveness, efficiency, relevance and coherence of the policy was overall considered positively. Stakeholders that are related to the agricultural sector and those who are involved in environmental conservation and restoration appeared to be the least positive about the impact of the rural development programmers. Groups that work on social cohesion are generally more positive about the effects of the rural development programmer. Furthermore, respondents (without any major difference by stakeholder group) concluded that the rural development programmes have added value and are able to address the economic, social and environmental needs of rural areas. Among citizens is thought that EU funding is better able to provide, long-term focussed and independent funding. Lastly, bureaucracy and administrative burden are seen as the main barriers in the application process and it could lead to a decrease in the uptake. Many respondents asked for simplification of the future policy design and administration.

Individual respondents represented 56% (403 respondents) of the total sample. More than half of the respondents are farmer or other supply chain actors (processors, retailers, wholesalers). Citizens of rural areas present one third of this group of respondents. Only 36% of the individual respondents are beneficiaries under the rural development programmes. Of this 36% most of the beneficiaries are farmers. Farmers are mostly beneficiaries of Axis 1 (competitiveness of agriculture and forestry) or multiple funding sources, whereas rural citizens are recipients under Axis 3 (economic diversification and quality of life) and LEADER. Most of the group of other individual respondents indicated that they were receiving funding under LEADER and Axis 2 (environment and land management). Claims that will be made later on in this report could be influenced by the uneven division between stakeholder groups.

Table 14. Distribution of individual respondents groups’

* Numbers do not add up, because of some of these respondents are also a fulfilling multiple roles

Source: Own elaboration on the basis of the public consultation

In total, 317 responses were received from professionals and organisations, which represents 44% of the total sample. Nearly half of this group were civil society organisations. One third of the respondents were from the private sector. Almost two third of the professionals and organisations that responded to the survey are beneficiaries, of which almost half of it is a civil society organisation. The source of the funding that is received by the professionals and organisations is more distinctive than in the case of the individuals. Public sector organisations such as local and regional governments often receive more funding under more than one axis. Almost half of the private sector companies receive funding under axis 1 and LEADER is the main source of funding for civil society organisations (77%).

Table 15. Distribution of professional and organisation respondents groups’

Source: Own elaboration on the basis of the public consultation

Individual respondents: since the survey allowed respondents to identify themselves as belonging to more than one category, farmers often also identified themselves as ‘individual citizens of a rural area’. Because this is a regular combination, the choice was made to include these cases only in the ‘farmer’ category. Other doublings with ‘individual citizens of a rural area’ were grouped in the category ‘individual citizens of a rural area’.

The general trends that can be observed in the answers are:

·Professional respondents and organisations respond more positively than individuals.

·Beneficiaries of rural development programme funding respond more positively than non-beneficiaries.

·Individual farmers and the group of professional farmers and farmers associations are more negative about the impact and relevance of rural development programme. They also indicate more often that they have observed no effect of the policy measures.

·Beneficiaries of Axis 2 funding are the least positive about the contributions of the rural development programme.

Competitiveness of agriculture and forestry

In general, more than 70% of the respondents within the different stakeholder groups think that rural development programmes have improved the competitiveness of agriculture. Comparing the different beneficiary groups, Axis 2 beneficiaries were the least positive: 17% of this group indicated that they thought rural development programme had a negative contribution.

On the rural development programme’s contributions to the competitiveness of the forestry sector, 29% of the respondents answered with ‘no opinion’. Another 20% perceived no effect of the measures. It might be more difficult to answer this question because the growth cycle of forest is a long-term process.

Improving of the environment, countryside and quality of life

Of the 11 environmental organisations that participated in the consultation, 30% thought rural development programme’s contributed negatively to the environment. This was the only question in which individual farmer were more positive about the rural development programme’s contributions than other individual citizens. It was also interesting to see that 18% of the Axis 2 beneficiaries gave a negative response, which is almost double the percentage of the other beneficiaries.

For Axis 2 beneficiaries, similar results were found in the responses to the questions on the RPD’s contribution to improving the countryside: 22% answered negatively. Public sector and civil society were very positive about the impact on the countryside, approximately 90% of these organisations were positive. The broad interpretation possibilities of the question could have led to differences in the results per stakeholder groups.

Considering the impact on quality of life, farmers are the least positive about the achievements under the rural development programme s: of the professional farmers, 22% thinks there is no effect and 10% think rural development programme has had a negative impact on their quality of life. Similar trends were seen for individual citizens in rural areas (15% and 10% respectively).

Diversification

Two questions were related to diversification. One focussed on the contribution of the rural development programme to encouraging diversification and the other on building local capacity for diversification. Considering the first topic, beneficiaries of Axis 3 funding were the most positive, together with the civil society organisations (> 80%). Axis 1 and 2 beneficiaries most often saw no effect of the measures. Local capacity for diversification received slightly different answers. More respondents saw no effect or perceived a negative contribution. Private sector organisations and individual farmers were the least positive about the impact.

Employment and Quality of life

Although the overall perception of the impact of the rural development programme of the local capacity to create employment was positive, still 21% of respondents thought that the programmes have had no effect. The stakeholder groups can be found in this group of respondents were individual respondents in general and actors form the private sector. The private sector gave far more negative responses that the other professionals and organisation, 9% compared to 2%.

Only 10% of the respondents indicated that the rural development programme did not provide value for money. The largest group of negative respondents were found in individual farmers (18%), private sector companies (14%), Axis 1 beneficiaries (16%) and Axis 2 beneficiaries (14%).

The rural development programme 2007-2013 was thought to be consistent with other EU funded interventions in the respondents’ areas. The private sector was the least positive stakeholder group: 19% of this respondent group thought the rural development programme were not consistent.

Similar results were found in the question on whether rural development programme fitted the needs of the area. 22% of the private sector companies responded that the programmes did not fit the needs. This was also the case for the individual farmers (21%). Considering the beneficiaries, Axis 1 and 2 beneficiaries were the most negative, 17% and 22% respectively.

Analysis of the open questions

Respondents were given the chance to share their personal views on the rural development programmes 2007-2013. The survey contained four open questions and a field in which respondents could add further comments of any kind. A majority of the respondents filled in the open questions and 144 respondents filled in the field for further comments. These comments were often an enforcement of statements that were given elsewhere in the survey. Others included specific local situations or problems with the implementation of the rural development programmes in specific Member States or regions.

Respondents were asked what the most essential benefits of EU financing were for rural development programmes that would not have been achieved by the Member States/regions acting on their own. A wide range of answers were given. In general, respondents are positive about the EU added value. Only a few respondents do not see the benefits of EU funding instead of Member State action, or have a feeling that the EU funding leads to a distortion of competition. A few times unfair practices with EU funding was mentioned.

In general, EU funding is considered a stable funding source, which also led to an increase in national funding through the co-financing principle. EU funding can also have a spillover effect and attract other developments in rural areas. Respondents indicated that it is important to them that EU funding is not influenced by local political dynamics and that there is more equality between the beneficiaries. Furthermore, the variety between projects that could be subsidised with EU funding was perceived higher than national funding.

The answers of the respondents that do think there is EU added value can be broadly split up in four categories: agriculture, environment, rural development, and local empowerment.

According to respondents, EU funding has stimulated modernisation and innovation in agriculture. Many respondents think that the member states on their own would not have made available (enough) funding to help farmers to invest in their companies. EU funding has stimulated agricultural sectors that otherwise would have had more difficulty to be self-sustaining, such as organic agriculture and agro-forestry.

Many respondents think that the EU funding has led to a better protection of the environment. There is a belief that national/regional government bodies would not have had the same priorities when acting on their own. Considering environment, climate action and biodiversity the respondents value the EU intervention and see the need to set more overarching goals.

Considering rural development, respondents see the EU funding as stimulance for beneficiaries and farmers to invest in their local areas, which would otherwise have been too costly. Some respondent’s belief that member states would focus their subsidies and investment too much on urban areas and economic development, whereas rural development programme is creating a more territorially balanced intervention. Investment in vital infrastructure was mentioned several times as something that would have gotten less funding through national investment programs.

Finally, many respondents see EU funding as a stimulator for local integration and partnerships. The bottom-up approach that is incorporated in LEADER, is useful to address specific needs of local areas. Many respondents appreciate this. The rural development programmes are perceived as solidarity measure within the EU as a whole and between the member states. Furthermore, respondents indicated that the funding stimulates cross-border cooperation and exchange of best practices.

Improving living conditions

Respondents were asked if they have suggestions on how future rural development programmes could further improve living conditions in rural areas. Again this had led to a broad variety in answers. Some of the topics that were often mentioned were: simplification and less administrative burden; focus on better connecting the rural areas with urban areas (both hard infrastructure as well as broadband); increasing the usage of the bottom-up approach; more focus on small and medium enterprises and farms; stimulating environmental conservation and restoration.

Simplification

Respondents were asked about their opinion on what could be simplified in order to improve the management and implementation of rural development programmes. The general trend in the answers includes a demand for simplification of the application procedure and the administration. Red tape seems to be a general point of frustration. Other things that respondents regularly mention are a need for more transparency of the application process and more information about the funds that are available to apply for.

Future application

On the question whether and why respondents would apply for future funds or not, most respondents indicate that they will. Many respondents mention that it is important for them that rural development will remain to be supported. The EAFRD is an important factor in the development of in rural areas. One of the barriers that is often pointed out is again the bureaucracy.

Uploaded documents

As stated above, 33 respondents have uploaded additional documents to the survey answers. Two documents were received outside the formal context of the survey, but by email. In total, 28 unique documents were received. A large number of documents address the future of the CAP post 2020 or focus on the period 2014-2020. Most likely this is due to the dominance of the topic in the CAP debate. As a result, these documents fall outside the scope of the evaluation study and are not included in the analysis of this report.

A few documents dealt with general topics related to agriculture and rural development. For example, two respondents uploaded a position paper on the lack of communication about the positive impact of agriculture on rural development concerning for example the ecosystem services delivered by agricultural landscapes.

Three documents did focus on the 2007-2013 period. The respondents gave information on specific issues with the implementation of rural development programmes and the impact in their own member state. For example, an evaluation report on the environmental impact of the rural development programme in Ireland, or the impact of the rural development programme on mountain farming in Austria. This is valuable information, but too specific to be incorporated in the general evaluation of the rural development programmes.

Annex 3. Data on implementation of rural development programmes by measure

Axis 1 - Competitiveness

Measure 111: Vocational training and information actions

In order to improve the competitiveness of the beneficiaries, a total budget of EUR 840.5 million has been spent on M111 by 26 Member States across 75 regions. This has resulted in:

·An output of 6 378 034 participants in training and 6 826 136 training days received. With more than 6 million participants in training, the measure reaches 28.7% of active farm workers in the EU as a whole.

·Result indicators for this measure are the number of participants passing by achieving certificate, degree or diploma and the number of participants implementing the achieved skills. In total 1 862 342 participants passed trainings by achieving a certificate, degree or diploma (on average 80 972 per Member State). Out of the participants, 1 987 552 implemented the achieved skills (on average 132 503 per Member State).

55% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 14% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 8% of the evaluation reports concluded the measure did not contribute. The increase in competitiveness was mainly attributed to an increase in the education level of farmers, which resulted in a better performance of their farms, including in terms of environmental management. The information judging the contribution of the measure is based on 75 reports of which 58 reported on the contribution of the measure.

Of those reports that provided a conclusion on M111, 71% stated a positive contribution. Based on these evaluations, it is concluded that the measure contributed to an improved competitiveness to a medium extent.

Measure 112: Setting up of young farmers

The majority (66%) of the programme level ex post reports identified a positive contribution to improving the competitiveness of beneficiaries, and that of the sector as a whole, because young farmers are more inclined to introduce new methods and techniques. In more than a third of these cases, the measure is also considered to have facilitated structural adjustment of farm holdings, and a number also reported positive environmental benefits. However, some reports identified significant deadweight. Overall M112 is considered as one of the most effective means of increasing competitiveness.

In order to improve the competitiveness of the beneficiaries, a total budget of EUR 3 271 million has been spent on M112 by 24 Member States across 69 regions. This has resulted in:

An output of 192 003 assisted young farmers and a total volume of investment of EUR 17.4 million. The assisted young farmers amount to 29.8% of the farmers in the EU in that age category. The total volume of investment is limited to around 0.1% of the total GVA.

The result indicator for this measure is the increase in total GVA in supported holdings/enterprises. A total increase of EUR 6.1 billion was reported.

Compared to the other measures, this measure is one of the most successful in increasing the GVA of the supported holdings/enterprises.

66% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 10% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 9% of the evaluation reports concluded the measure did not contribute. The increase in competitiveness resulting from this measure was mainly attributed to the modernisation process that was started when young farmers took over businesses. The above information is based on 59 reports.

Of those reports that provided a conclusion on M112, 78% stated a positive contribution. Based on these evaluations, it is concluded that the measure contributed to an improved competitiveness to a great extent.

As far as other, indirect effects are concerned, positive effects on the environment are the most prominent (28%). This is because young farmers have more knowledge on environmentally friendly practices. Young farmers applied more environmentally friendly practices than older farmers. This resulted in an improvement in natural environmental conditions. Also effects on increase in available jobs and on reduction in population decrease were noted.

Measure 113: Early retirement of farmers and farm workers

Less than a third of the ex post evaluation reports found that the measure made a positive contribution to competitiveness. A similar proportion reported other benefits such as improved land management and/or reduction of land abandonment. Reasons why the measure was not considered more successful include low levels of funding and uptake. Compared to other measures, support for early retirement is considered to have made only a marginal contribution to increasing GVA on the supported holdings.

A total budget of EUR 2.7 billion has been spent on M 113 by 16 Member States across 51 regions.

31% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 19% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 21% of the evaluation reports concluded the measure did not contribute. By facilitating early retirement of farmers, farms and farmland becomes available for younger farmers. Hence, these were able to improve their competitiveness by profiting from economies of scale through the expansion of their company, and by improving old farms through the introduction of modern farming techniques. The information is based on 51 reports of which 40 reported on the contribution of the measure.

Of other effects observed, mainly improvement in land management (33%) was the most important.

Measure 114: Use of advisory services by farmers and forest holders

The majority of advisory services provided focussed on compliance with EU standards rather than competitiveness. Implementation was relatively low, reaching less than 2% of EU farmers, and 0.4% of forest holders. This is perceived to be due to lack of interest in the services on offer, in part because of insufficient focus on competitiveness. However, environmental improvements were identified in one third of evaluations.

M114 aimed to increase the beneficiaries’ competitiveness by supplying support to providing advisory services that directly improved operational as well as agricultural practices. A total of EUR 133 million was spent by 20 Member States in 55 regions. The overall results were:

178 498 farmers, representing 2% of the EU total of farmers in 2013 received advisory services under this measure. Additionally, 2 406 forest holders, representing 0.4% of the EU total of forest holders in 2013 received advisory services.

A total GVA increase of EUR 642 million was reported. Compared to the other measures and based on this indicator M114 measure was only moderately successful in creating value added. 41% of the evaluations that provided a conclusion on this measure found that it had a positive contribution to competitiveness.

Only 37% of the evaluations identified other effects, where they were identified they were mainly environmental.

Measure 115: Setting up of farm management, farm relief and farm advisory services, as well as of forestry advisory services

M115 was implemented in 7 Members States and 27 regions with total budget of EUR 31.7 million in total. The measure concerned setting up advisory services to assess farm performance and identify possible improvements to increase competitiveness. It was generally considered to make a positive contribution to increasing the beneficiaries’ competitiveness, particularly for small and medium sized farms that did not previously have access to this type of expert support. In some cases too great a focus on compliance issues, rather than competitiveness, limited the attractiveness of the measure to potential beneficiaries. Complicated administrative procedures were seen as the main constraint to a more effective implementation.

For 41% of the evaluation reports, this measure had a positive contribution to competitiveness of the beneficiaries. This positive effect was attributed to a more efficient and sustainable use of resources in 10% of the cases, since the advisory service offered the farmers a system to assess farm performance and pinpoint possible improvements to increase competitiveness. 60% of the positive reports found that M115 helped farmers improve both their techniques and management, which ultimately led to a better market position. In addition, 10% of the positive reports mentioned that the measure helped to commercialize ecological products.

30% of the ex post evaluation reports mentioned a limited contribution to competitiveness under M115. This was attributed primarily to administrative issues in the implementation, which was halted by complicated procedures and paused operation. In some cases, the offered services were not considered to be oriented towards to farmer’s interests, which had a demotivating effect on the applicants.

22% of the rural development programmes reported no contribution to competitiveness. In these cases, 66% reports explained that the measure’s focus on the promotion of compliance to statutory requirement meant there were no competitiveness impacts. For 33% of the reports, the measure was not fully executed, which also prevented any impacts from materializing.

Based on these evaluations, it is assessed that the measure contributed to an improved competitiveness only to a limited extent.

Measure 121: Modernisation of agricultural holdings

This measure was implemented in all rural development programmes (88), and provided 11.7 billion EUR of public support, reaching 4.3% of farm holdings across the EU. It was found to have increased competitiveness, production and labour efficiency, and to have introduced new methods and products. Where environmental effects were reported, the impact was considered positive in 86% of cases and negative in only 8%.

467 324 farm holdings were supported. 166 749 holdings introduced new techniques and/or products. A total increase in the gross value added in the supported holdings of EUR 19.3 billion was observed. Compared to the other measures, this measure is one of the most successful in supporting the introduction of new products and/or new techniques and in increasing the GVA of the supported holdings/enterprises.

71% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 10% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 5% of the evaluation reports concluded the measure did not contribute. The increase in competitiveness resulting from this measure was mainly attributed to its focus on modernisation.

When studying other effects, effects on the environment were referred to in 27% of the ex post evaluations. These effects relate to emissions, energy and the reduction of soil erosion.

Measure 122: Improvement of the economic value of forests

Although the total support granted through this measure was small, it was found to have a positive impact in the majority of cases where it was implemented, generating a considerable increase in the GVA of the forestry sector (almost three times the investment provided). Environmental improvements and contributions to climate change mitigation were also identified, as were positive contributions to diversifying the rural economy, and improving the conditions of forestry workers (increased safety and reduced working hours).

M122 had a level of expenditure of EUR 309 million dedicated to improving competitiveness through the improvement of the economic value of forests. This budget was spent by 17 Member States in 49 regions. 26 322 holdings received investments support under this measure, totalling an investment of 936 million. 7 573 holdings introduced new products and techniques. It generated an increase in GVA of 2 754 million, making it one of the more successful measures in generating value added and representing 11% of the total GVA for forestry for the time period.

43% of the reports found that the measure had a positive contribution to competitiveness, 27% found that the contribution was limited, and 12% found that there was no contribution. Of those reports that provided a conclusion on M122, 53% stated a positive contribution hence it can be concluded that that the measure contributed to an improved competitiveness to a medium extent.

25% of the evaluations also mentioned additional effects, mainly contribution to the sustainable management of forestry.

Measure 123: Adding value to agricultural and forestry products

This measure, which was widely implemented (85 rural development programmes in 25 Member States) was considered to improve the competitiveness of the industry beneficiaries in two thirds of reports, making a limited contribution in a further 20%. It improved product quality, introduced new processes and reduced costs. 19% of the reported increase in GVA of the sector over the period could be attributed to support through this measure. Positive effects on the environment (better resource management, more environmentally friendly processes) and employment were also found in some cases.

M123 had a total level of expenditure of EUR 4 413 million dedicated to adding value to the agricultural and forestry products). 28 265 enterprises were supported; The total volume of investment under M123 is EUR 22 billion, increase in GVA in supported enterprises under M123 is EUR 30 billion, representing 19% of total GVA increase for the sector in the time period; 14 484 holdings with new products and / or techniques introduced under M123 were registered.

66% of the evaluation reports found that the measure had a positive contribution to competitiveness, 20% found that the contribution was limited, and 1% found that there was no contribution.

Of those reports that provided a conclusion on M123, 76% stated a positive contribution. Based on these evaluations, it can be concluded that the measure contributed to an improved competitiveness to a great extent.

Additional effects mentioned in the evaluation reports were mainly maintenance/creation of jobs.

Measure 124: Cooperation for development of new products, processes and technologies in the agriculture and food sector and in the forestry sector

The most widely reported effects of this measure were increased levels of innovation and more productive value chains. Although accounting for a very small proportion of expenditure, it accounted for 6% of reported introductions of new techniques and products. In many cases it was considered that the contribution to competitiveness will only become significant in the medium to long term.

The level of expenditure under M124 was of a total of EUR 234.3 million spent on the cooperation for development of new products, processes and technologies. This amount was spent by 14 Member States in 51 regions. The overall results were as follows:

A total of 5112 cooperation initiatives were supported as a result of M124.

The number of enterprises introducing new techniques and / or products under M124 was 12 972. The measure contributed 6% of the holdings with new techniques and products under the entire rural development programmes.

The result indicator of increase in GVA in supported enterprises under M124 is EUR 122.6 million.

33% of the reports found that the measure had a positive contribution to competitiveness, 26% found that the contribution was limited, and 11% found that there was no contribution. Of those reports that provided a conclusion on M124, 47% stated a positive contribution. Based on these evaluations, it can be concluded that the measure contributed only to a limited extent to an improved competitiveness.

Most quoted other effects were mainly environmental; the fostered cooperation resulted in knowledge transfer on more sustainable practices.

Measure 125: Improving and developing infrastructure related to the development and adaptation of agriculture and forestry.

Investments supported through this measure were found to have a positive effect on competitiveness in the majority of cases, particularly through reducing transport costs along the value chain (improved roads) and reducing input costs (irrigation infrastructure). The investments in irrigation also lead to reduced water consumption and therefore environment/climate benefits. Improved accessibility also improved the quality of life of rural residents and increased tourism activity.

EUR 4 317 million was devoted under this measure to improving and developing infrastructure related to the development and adaptation of agriculture and forestry. This budget was spent by 24 Member States in 78 regions.

In the output indicator of total volume of investment, M125 received EUR 10 billion;

The output indicator of number of operations supported under M125 reached a total of 56 779 operations; For the result indicator of increase in GVA in supported holdings, M125 generated a value added of EUR 7 billion, making it one of the most successful measures in generating value added.

59% of the reports found that the measure had a positive contribution to competitiveness, 18% found that the contribution was limited, and 6% found that there was no contribution. Where there was a limited contribution, this was often due to the objective of the infrastructure, which was not necessarily improving competitiveness, but bringing basic infrastructure to regions that lacked it.

The information is based on the 59 ex post evaluation reports that reported on the contribution of the measure, out of the total of 71 reports.

Of those reports that provided a conclusion on M125, 71% stated a positive contribution. Based on these evaluations, it can be concluded that the measure contributed to competitiveness to a medium extent.

As additional effects, a positive environmental effect was mentioned as a result of new irrigation techniques allowing for a better use of water resources and reducing overall water consumption.

Measure 126: Restoring agricultural production potential damaged by natural disasters and introducing appropriate prevention action

This measure was implemented in only 7 Member States (30 rural development programmes), mostly to restore damaged production potential. Where it was used, the effect was generally considered positive. Prevention measures supported (such as flood defences) were considered positive not only for direct beneficiaries, but for industry, local communities and the environment.

A total budget of EUR 648.5 million has been spent on M126 to support 2.9 million hectare of damaged agricultural land. 42% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 17% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 38% of the reports concluded the measure did not contribute. In 67% of the positive cases the increase in competitiveness was attributed to the ability to restore factors of production. Prevention of economic losses and technical improvement were also pointed at as results of the measure which have increased.

Based on these evaluations, it can be assumed the measure contributed to an improved competitiveness to a limited extent.

Measure 131: Helping farmers to adapt to demanding standards based on Community legislation

Both the coverage and expenditure for this measure were very low, concentrated in a small number of rural development programmes. Insufficient evidence was found to draw general conclusions as to its effects. However, some specific benefits were identified, such as reducing the cost of electronic ear-markers for sheep and goats in Hungary, which helps farmers to comply with the relevant requirements.

A total budget of EUR 62.1 million has been spent on M131 by 11 Member States across 20 regions, supporting 29 644 beneficiaries. This is a marginal share (0.1%) of the total amount of active farmers within the EU; the total increase in GVA in supported holdings/enterprises was reported in only 1 of the Member States that implemented the measure; hence no general conclusion concerning the increase of GVA in supported holdings/enterprises can be drawn. 24% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 24% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 24% of the evaluation reports concluded the measure did not contribute.

The measure helped to fulfil legal requirements based on Community legislation. This has increased the competitiveness of beneficiaries. On the other hand, the measure is seen as compensation for extra cost, and thus does not increase the competitiveness of beneficiaries. Of those reports that provided a conclusion on M131, 33% stated a positive contribution. Based on these evaluations, the measure can be considered to contribute to an improved competitiveness only to a limited extent.

Measure 132: Supporting farmers who participate in food quality schemes

This measure was implemented by 15 Member States (47 rural development programmes), with a very small budget share (EUR 91.2 million). A positive contribution to competitiveness and changing production methods was reported in over a third of the relevant evaluation reports, but the effect was limited. A similar proportion of reports identified environmental benefits, principally reductions in chemical use and emissions.

578 983 farm holdings that participated in a food quality scheme were supported. In comparison with the total number of holdings with livestock in the EU, around 8% of the total holdings has participated in food quality schemes; 37% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 22% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 20% of the evaluation reports concluded the measure did not contribute. Results were limited due to low amounts of funding. One ex post evaluation also pointed to a 100% deadweight loss.

Overall, it can be concluded that his measure contributed to an improved competitiveness to a limited extent.

Measure 133: Supporting producer groups for information and promotion activities for products under food quality schemes

Another “small” measure, implemented in 16 Member States across 41 regions and accounting for less than 1% of expenditure in all cases except one. A total budget of EUR 128.4 million has been spent for 111 12 supported actions. Overall the measure had some positive effects in promoting awareness of quality-labelled products, but effects were small due to low budget and limited targeting.

44% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 15% of the ex post evaluations concluded the measure’s effects on competitiveness as limited and 15% of the evaluation reports concluded that the measure did not contribute. Positive contributions were attributed to an increase in awareness of local produce, which boosted the competitiveness of the beneficiaries. Limited contributions have occurred due to various reasons like a low budget or a low implementation rate. Based on the evaluations, it can be stated that the measure contributed to an improved competitiveness to a limited extent only.

Measure 141: Supporting semi-subsistence agricultural holdings undergoing restructuring

This measure was implemented in 8 Member States where semi-subsistence farms constitute a significant proportion of farm holdings. Positive effects, including increased competitiveness, higher value products, and social benefits were reported in 80% of the relevant evaluation reports.

In order to improve the competitiveness of the beneficiaries, a total budget of EUR 772 million has been spent on M141. This has resulted in 66 051 semi-subsistence farm holdings supported and 3 585 new farms entering the market.

40% of the ex post evaluation reports found that the measure contributed positively to improving the competitiveness of the beneficiaries. 40% of the ex post evaluations concluded the measure’s effects on competitiveness were limited and 20% of the evaluation reports did not give a clear conclusion. Based on these evaluations, the measure is considered to contribute to an improved competitiveness to a medium extent.

Measure 142: Supporting setting up of producer groups

This measure, which was implemented in 11 Member States or regions with total budget of EUR 235 million, is considered one of the least effective, with no reports concluding that the measure made a significant positive contribution to competitiveness. A limited effect was identified in 44% of cases, for example increased sales volumes, and improved quality of produce or better bargaining power.

The aim of this measure was to improve the market efficiency of the agricultural sector by encouraging and supporting the setting up of Producer Groups. The majority of evaluation reports (44%) found that the measure had only a limited contribution to the improvement of competitiveness, some pointed to a negative effect.

Measure 143: Provision of farm advisory and extension services in Bulgaria and Romania

This measure was only available to two Member States, and had a total budget of 10 million EUR. Evaluation results were only available for Romania which found that it had only a limited contribution to economic growth and productivity, although it helped improve management capacity and knowledge of EU standards.

Measure 144: Holdings undergoing restructuring due to a reform of a common market organisation

This measure specifically aimed to support tobacco farms following the reform of the common market organisation and was implemented in 5 Member States (11 rural development programmes). It does not appear to have been effective, with a positive impact identified in only one report, achieved through the implementation of associated business plans.

The total level of expenditure under M144 was of EUR 180.8 million, spent on supporting holdings undergoing restructuring due to a reform of a common market organisation. Half of the evaluation reports found there was no contribution to competitiveness, only one found a positive contribution. There were no indirect effects noted. Hence, the contribution of this measure to competitiveness has to be judged as extremely limited.

AXIS 2 Environment and the countryside

Measure 211: Natural handicap payments to farmers in mountain area

M 211 was implemented in 15 Member States in a total of 60 regions, with a total budget of EUR 7 391.1 million. The main objective was to compensate farmers for the additional costs and income losses arising from the difficulties of agricultural production in mountain areas. The aim is to maintain the countryside through continual use of agricultural land and to promote the systems of sustainable agricultural production, thereby supporting the improvement of the environment.

The measure resulted in 1 049 665 supported farm holdings and 16 052 054 hectares covered (one Member State implementing the measure did not report the output indicators). This amounts to 57.4% of the UAA in the less-favoured mountain areas of the Member States that have offered the measure and to 56.7% of the UAA in mountainous areas in the EU as a whole.

Over two thirds of the reports stated that it made a positive contribution to the environment, principally through maintaining extensive agricultural systems in areas where appropriate land management is important for biodiversity and High Nature Value habitats. It was also recognised as helping retain population and employment in fragile rural areas.

Measure 212: Payments to farmers in areas with handicaps, other than mountain areas

A positive contribution of M212 to the environment was reported in almost half the reports, mainly through avoidance of land abandonment. A further 25% indicated some contribution to the environment, with the benefits limited due to lack of environmental requirements associated with the payments. There was no obligation to farm sustainably/extensively, and intensification and loss of permanent grassland was observed in some cases.

In order to compensate farmers for difficulties of agricultural production in less favoured areas other than mountain areas, 27 Member States implemented M212, covering 75 regions and spending a total budget of EUR 7 681.4 million. On average, Member States invested 10% of their total EAFRD expenditure in M 212, with wide disparities. This has resulted in:

An output of 1 843 831 supported farm holdings and 36 836 442 hectares covered (two of the 27 Member States that implemented the measure did not report on the Output Indicators). With about 37 million hectares of arable land, the measure covers 59.2% of the UAA in the less-favoured areas other than mountain areas of the Member States that have offered the measure.

In 46% of the ex post evaluation reports the general contribution of M212 to improving the environment was assessed as positive, 25% stated a limited effect and 6% saw no contribution. The positive effects on the environment resulting from this measure were mainly attributed to its support to maintain agricultural cultivation on sites, which otherwise would be threatened by land abandonment and a possible degradation of natural values. Some reports stated that the measure has made a positive contribution to the promotion of extensive management practices, while others concluded that it was not expected to have a significant impact on promoting extensive farming. However, the lack of specific environmental protection requirements under M212 and M211 was a reason for limited ratings. Overall, it is considered that the measure contributed to the improvement of the environmental situation to a medium extent.

Measure 213: Natura 2000 payments and payments linked to Directive 2000/60/EC

This measure was recognised as supporting farmers’ incomes in the 13 Member States (28 rural development programmes) where it was implemented, by compensating them for financial losses due to using farming practices beneficial for the designated habitats/species. Whilst the management practices are compulsory, even without these payments, positive effects on biodiversity were reported due to improved understanding and acceptance of Natura 2000 requirements and high coverage of target areas.

In order to compensate farmers for disadvantages caused by Natura 2000 regulations or the Water Framework Directive in agricultural areas, 13 Member States implemented M213, covering 28 regions and spending a total budget of EUR 267.5 million. It has resulted in 75 199 supported farm holdings. The measure covers 15.3% of the UAA in Natura 2000 areas of the Member States that have offered the measure. In 28% of the ex post evaluation reports the general contribution of M213 to improving the environment was assessed as positive, while 31% stated a limited effect and 17% saw no contribution. 10% of the evaluations of the measure indicated that the effect was not clear.

The assessments have described in several cases that the measure itself did not have any impact and/or that the effects are exclusively due to the underlying mandatory provisions of the Natura 2000 network, i. e. are indirect in nature. It was stressed that the rules laid down in the respective EU directives must be complied with, even without any support under M213. However, several reports concluded that the measure has improved the knowledge and understanding of farmers and beneficiaries in relation to the regulatory requirements and hence contributed to the conservation objectives pursued by the Natura 2000 network. Several reports have highlighted the option of a combination of Natura 2000 compensation with other measures such as M214 – in particular regarding the potential synergies in terms of environmental impact.

Based on the evaluations, the measure is considered to have contributed to the improvement of the environmental situation to a limited extent.

Measure 214: Agri-environment payments

All rural development programmes had to include this measure, and it accounted for the largest proportion of expenditure. Almost 90% of reports giving a clear judgement on this measure concluded that its implementation made a positive contribution to the environment, through both the extent of area covered by commitments and their effectiveness. Biodiversity and High Nature Value farming were identified as most significantly affected (69%), improved soil quality in 66% and water quality in 61% of reports. This measure was judged as having made the largest contribution to mitigating climate change, through reducing GHGs, sequestering carbon and reducing fire risks. A variety of indirect effects on quality of life, retention of population and employment/diversification opportunities were also identified.

Member States spent a total budget of EUR 23 619.4 million on different operations aimed at improving the environmental situation. Most regions have implemented a variety of sub-measures with different focuses. On average, Member States invested almost 26% of their total EAFRD expenditure in M214. This has resulted in 1 521 872 farms that were supported and 12 237 other land managers. The measure concerned 968 086 livestock units; as well as

-42.5 million ha under land management contributing to biodiversity,

-35.7 million ha contributing to water quality,

-36 million ha contributing to soil quality,

-25.5 million ha contributing to mitigating climate change,

-19.6 million ha to the avoidance of marginalisation and land abandonment.

Since several actions are possible on the same area, a comparison with reference values is not possible.

Of those reports that provided a clear conclusion on M214 (i.e. “positive”, “limited” or “no contribution”), 87% stated a positive contribution. Based on these evaluations, it can be stated that the measure contributed to the improvement of the environmental situation to a high extent.

Measure 215: Animal Welfare

This measure was implemented in 11 Member States (30 rural development programmes), with a total EAFRD expenditure of EUR 1 billion. Its main objective was to improve animal welfare. It was perceived as making a positive (34%) or limited (17%) contribution to the environment in half the cases. The benefits identified included extensification and lower livestock density, with effects on pasture species composition, reduced N and P water pollution and improved soil structure and fertility. In a small number of cases, improved animal welfare was identified as contributing to improved animal health and production, and hence competitiveness.

The main topics in the 29 reports – when explained in more detail – address whether production systems (e.g. support of free-range husbandry, ecological livestock production systems) or animal groups were targeted (dairy cows, pig breeding, poultry farming, aviculture etc.). 79 435 farm holdings were supported and 143 099 contracts made. The measure reached 1.7% of the total number of holdings with livestock in the Member States that offered the measure. This corresponds to 1.1% of all holdings with livestock in the EU. The impact of the measure is therefore limited.

Since 28% of the reports attributed no contribution of M215 to an improvement of the environmental situation, another 7% gave no information on the topic and 14% was unclear on the contribution to the environment, the assessment is indicative only.

Measure 216: Support for non-productive investments

Support provided under this measure covered a wide variety of non-productive investments. A positive environmental contribution, particularly to biodiversity, landscape, HNV and Natura 2000 areas, was reported in 43% of reports (more than half of those providing a judgement), and a limited contribution in a further 27%. Where the benefits were considered “limited”, constraining factors mentioned included late implementation of the measure, low uptake, and the small scale of supported projects.

In order to support non-productive investments in farm holdings, a total budget of EUR 621.0 million was spent on M216 by 16 Member States across 53 regions. This has resulted in 44 294 farm holdings supported and a total investment volume of EUR 1 190 million.

Result indicators are reported on Axis level, concerning the number of hectares supported by the measure reported by 13 Member States. The strongest impact is reported for soil quality, marginalisation and biodiversity. However, it is not clear, how the relatively high numbers of hectares for these environmental aspects are related to non-productive investments.

According to the synthesis, 43% of the reports attribute a positive contribution to the measure. A rather large number of aspects were considered as improvement for the environment and the landscape. The conservation and development of habitats and biodiversity were mentioned several times. Further topics were the reduction of erosion, stability of slopes, better soil quality management, and improvement of water quality, preventing natural hazards and increasing carbon sequestration. Landscape-related topics were the conservation and improvement of the scenery.

A limited contribution of the measure is found in 27% of the reports. 8% of the reports attributed no contribution of the measure, 20% did not deliver any information on the topic (N/A) and 2% provided other classifications (not clear). It is concluded that the measure has contributed to a medium extent to improving the environmental situation.

Measure 221: First afforestation of agricultural land

A total budget of EUR 1 586 million has been spent on M221 by 20 Member States across 63 regions for first afforestation of agricultural land. This resulted in 203 944 hectares of afforested land in 42 531 supported actions. The measure reached 0.1% of the total UAA or 0.2% of the total forest area across the Member States that offered the measure.

37% of the ex post evaluation reports assessed a positive contribution of the measure to the improvement of the general environmental situation (mostly regarding biodiversity, mitigation of climate change and water quality). This was largely attributed to the diverse and various ecological functions of forests and hence the afforested areas were often assessed positively without further consideration. However, as 39% of the ex post evaluation reports did not include a detailed examination of the measure's impact on the environmental situation, the available information covers the regions which implemented the measure to a small extent only.

Breaking down the assessment of the general environmental situation in different topics, 43% of the reports noted a positive contribution to biodiversity, 35% on water quality, and 46% on mitigating climate change and 26% on soil quality but only 7% on avoidance of land abandonment.

In summary, the measure can contribute to improving the environmental situation depending on the utilisation, design and initial local environmental situation. It can be stated that the measure contributed to the improvement of the environmental situation to a medium extent.

Measure 222: First establishment of agroforestry systems on agricultural land

A total budget of EUR 1.5 million has been spent on M222 by 5 Member States across 8 regions, which has resulted in 2 905 hectares of supported utilised agricultural area (UAA) in 291 supported actions. In comparison with other measures, only a very small share of the UAA was supported with this measure (0.0017% on EU level). Very limited consolidated information is available concerning its effects. Specific comments make reference to the new habitat increasing biodiversity, reducing chemical inputs, and improving soil and water quality. Due to the low utilisation of the measure in the rural development programmes and the limited impact on the individual aspects of the result indicator, it is assessed that M222 contributed to the improvement of the environmental situation to a very limited extent.

Measure 223: First afforestation of non-agricultural land

A total budget of EUR 168 million has been spent on M223 by 10 Member States across 32 regions. This resulted in 84 265 hectares of afforested land in 10 680 supported actions. 27% of the ex post evaluation reports assessed a positive contribution of the measure to the improvement of the environmental situation (mostly regarding the mitigation of climate change). However, detailed explanations were rarely presented, but the very broad impact of the measure on various environmental aspects has been emphasised and reference has been made to the in-depth analyses of M221; No or limited contributions of the measure were assessed in 23% of the cases. These evaluations are based on the very low utilisation and area coverage of the measure and therefore not on its design. In summary, the measure can contribute to improving the environmental situation depending on the utilisation, design and initial local situation, yet the measure contributed to the improvement of the environmental situation to a limited extent.

Measure 224: Natura 2000 payments – forestry

Although this measure was implemented on a minor scale in only 11 Member States (13 rural development programmes) and spending a total budget of EUR 74.3 million, its environmental impact was assessed as positive (76% of reports, albeit 42% qualifying the impact as limited, due to its nature as compensation for compulsory management practices). It is considered to preserve high quality habitats, to increase awareness and improve environmental management.

On average, Member States invested less than 0.25% of their total EAFRD expenditure in Measure 224. This has resulted in 14 391 supported forest holdings and 278 975 hectares woodland covered. The measure covers 2.3% of forests in Natura 2000 areas of the Member States that have offered the measure. This corresponds to 0.7% of forests in Natura 2000 in the EU as a whole;

On the one hand, the measure was attested to positively contribute to improving the environmental situation (33%), the justification being mainly based on the achievement of the target values and not taking a closer look at the individual environmental aspects. Some of the changes that can also be seen in the forest (e.g. deadwood content), which are actually due to higher-level regulations and not the measure itself, are mentioned here.

On the other hand, these regulation-induced changes were taken up in the second group of reports, in which the contribution to the improvement of the environmental situation was evaluated as limited (42%). Since the restrictions and management requirements of the Birds and Habitats Directives are to be complied with even without funding via M224, this measure has generally only had a limited additional effect on improving the environmental situation.

Measure 225: Forest-environment payments

Whilst implemented in 13 Member States (27 rural development programmes), this measure only accounted for 70 million EUR. Where it was implemented it was generally considered positive, but the very small coverage limited results. Much of the area covered was in Natura 2000 areas, so supported important habitats.

This has resulted in an output of 0.4 million hectare supported forest area and a total of 12 000 management contracts. 39% of the ex post evaluation reports found that the measure contributed positively to improving the environmental situation. 30% concluded the measure’s effects on the environment were limited and 4% concluded the measure did not contribute. In 78% of the positive cases the improvement of the environmental situation was attributed to higher ecological and biodiversity values. Improved soil quality and improved conditions regarding climate change were also pointed at as results of the measure which have improved the environmental situation.

Based on these evaluations, it can be plausibly considered that the measure contributed to the improvement of the environmental situation to a medium extent.

Measure 226: Restoring forestry potential and introducing prevention actions

This measure was assessed as highly positive for the environment, mostly through preventing fires and flooding. These actions improved biodiversity, soil and water quality, and mitigated climate change.

In order to improve the environmental situation, a total budget of EUR 1 542.1 million has been spent on M226 by 16 Member States across 56 regions. 10 million hectare of damaged forests were supported (a total of 77 359 actions).

62% of the ex post evaluation reports found that the measure contributed positively to improving the environmental situation. 15% of the ex post evaluation reports concluded the measure’s effects on the environment were limited and 9% of the evaluation reports concluded the measure did not contribute. In 78% of the positive cases the improvement of the environmental situation was attributed to the fields of fire damages and prevention. Improved water quality and flood mediation and biodiversity were also pointed at as results of the measure which have improved the environmental situation. Based on these evaluations, it is considered that the measure contributed to the improvement of the environmental situation to a medium extent.

Measure 227: Support for non-productive investments (in the forest sector)

The results of the implementation of this measure (13 Member States and 68 rural development programmes spending EUR 698.6 million) will become more evident over the medium-long term, making it difficult to evaluate clearly at the end of the programming period. However, the ex post evaluation reports consider that the contribution is positive, particularly for biodiversity and nature conservation, forest health and resilience against risks.

M227 has resulted in an output of an investment volume of EUR 1.7 billion and a total of 131.555 forest holders supported.

52% of the ex post evaluation reports found that the measure contributed positively to improving the environmental situation. 17% of the ex post evaluation reports concluded the measure’s effects on the environment were limited and 5% of the evaluation reports concluded the measure did not contribute. In 56% of the positive cases, the improvement of the environmental situation was attributed to improved biodiversity and nature conservation. Environmental improvements in general and forest health improvements and improved forest stability towards hazards were also pointed at as results of the measure which have improved the environmental situation.

Based on these evaluations, we assess that the measure contributed to the improvement of the environmental situation to a medium extent.

Axis 3 - Diversification of the rural economy and quality of life in rural areas

Measure 311: Diversification to non-agricultural activities

This measure was implemented by 20 Member States (65 rural development programmes) and was considered to contribute positively to the economic performance of beneficiaries in 52% of cases, rising to 90% when “limited” contributions are taken into account. 3% found no contribution. A wide range of activities were supported, including rural tourism, horse livery and renewable energy production. It appears to have been relatively effective in increasing GVA and jobs on the beneficiaries’ farms.

A total budget of EUR 1.1 billion has been spent on M311 by 20 Member States across 65 regions. This has resulted in 41 940 beneficiaries that received support for efforts to diversify. The result indicators for this measure are the increase in total non-agricultural GVA in supported businesses and the total number of jobs created. The increase in non-agricultural GVA in supported businesses is EUR 563.6 million. The total number of jobs created is 27 881.

Through the diversification, the measure contributed to the stabilisation of farms, enabled farmers to increase their overall performance, assisted farm households to maintain or increase their income, supported farms to maintain employment or even to create new jobs. Based on these evaluations, the measure is considered to have contributed to improving the economic diversification of the beneficiaries to a medium extent.

Measure 312: Business creation and development

A total budget of EUR 1.4 billion has been spent on M312 by 21 Member States across 46 regions. This has resulted in 74 138 supported micro-enterprises. The result indicators for this measure are the increase in total non-agricultural GVA in supported businesses and the total number of jobs created. The increase in non-agricultural GVA in supported businesses is EUR 823.7 million. The total number of jobs created is 68 843. 42% of the ex post evaluation reports found that the measure contributed positively to improving the economic diversification of the beneficiaries. 36% concluded the measure’s effects on the economic diversification of the beneficiaries were limited and 2% concluded the measure did not contribute. The measure contributed to improving the economic diversification of the beneficiaries to a medium extent.

Measure 313: Encouragement of tourism activities

This measure is perceived more positively than either M311 or M312. However, its efficiency appears to be considerably lower when considered as increased GVA or jobs created in relation to the investment made. The positive assessment may be linked to indirect effects (such as spending by additional tourists in the area, impact on other touristic infrastructure).

In order to improve the economic diversification of the beneficiaries, a total budget of EUR 887.9 million has been spent on M313 by 22 Member States across 67 regions. This has resulted in 24 518 new tourism actions that received support. The increase in non-agricultural GVA in supported businesses is EUR 350.4 million. The total number of jobs created is 17 578. 48% of the ex post evaluation reports found that the measure contributed positively to improving the economic diversification of the beneficiaries. 38% of the ex post evaluations concluded the measure’s effects on the economic diversification of the beneficiaries were limited and 10% of the evaluation reports concluded the measure did not contribute. The encouragement of tourism activities has led to substantial increases of the revenues from new agricultural products and services. Based on these evaluations, it is considered that the measure contributed to improving the economic diversification of the beneficiaries to a limited extent.

Measure 321: Basic services for the economy and rural population

This measure was implemented in 23 Member States (70 rural development programmes). It supported improved services for more than 45 million rural inhabitants, and provided internet coverage to an additional 16.6 million people living in rural areas. Improvements in quality of life highlighted in the reports focussed on social infrastructure such as day care, health and community facilities, and digital infrastructure. Smaller impacts were reported on water/waste water, electricity and roads. The overall assessment of this measure is positive.

Measure 322: Village renewal and development

This measure was implemented in 22 Member States (50 rural development programmes). The evaluation reports identified its effects on quality of life in rural areas as strongly positive, covering both physical infrastructure such as meeting places, and intangible aspects such as social cohesion and local/regional identity. The projects implemented also generated additional effects such as new employment, and counteracting depopulation.

Measure 323: Conservation and upgrading of the rural heritage

This measure was implemented in 22 Member States (70 rural development programmes), and is also considered to have had significant positive impact on quality of life, improving services for 66.4 million rural residents.

Measure 331: Training and information

Expenditure on this measure was much lower than for investment measures within Axis 3, although it was implemented by 13 Member States (32 rural development programmes). The evaluation assessments were mixed, with only 17% identifying a positive contribution to economic diversification and 25% of quality of life, with 36% a limited contribution to economic diversification and 22% to quality of life. This may be because the main activity supported was training to increase human capital, the benefits of which may only become evident in the long term, or in conjunction with other activities, such as business start-up.

Measure 341: Supporting skill acquisition for developing local strategies

This measure was implemented in 15 Member States (33 rural development programmes) and was intended to develop human capacity to develop and implement local development strategies. The effects of this measure were assessed as much more positive than those of M331 (32% identified positive contributions to economic diversification and 44% to quality of life, with 24% and 18% of reports respectively identifying further limited contributions). The findings focussed on the impact of the networks and relationships created, and the enhancement of collective capacity to implement LEADER.

Annex 4. Intervention logic by axis

Axis 1. Improving competitiveness of the agricultural sector

Axis 2. Improving the environment and the countryside

Axis 3. Quality of life in rural areas and diversification of the rural economy

Axis 4. LEADER

(1)      Article 86 of Council Regulation (EC) No 1698/2005, OJ L 277, 21.10.2005, p. 1–40.

(2)       Synthesis of Rural Development Programmes (RDPs) ex-post evaluations of period 2007-2013

(3)      COM (2018) 392 final.

(4)    European Commission, A long-term Vision for the EU’s Rural Areas, COM(2021)345 final, EUR-Lex 52021DC0345 - EN - EUR-Lex (europa.eu) .

(5)    Council Decision 2006/144/EC.

(6)    Including amounts arising from the application of the modulation system introduced in the 2008 CAP health check.

(7)    The 27 countries that were member of the European Union in 2006.

(8)     Rural development in the EU statistical and economic information: report 2009 .

(9)    See footnote 6.

(10)     https://ec.europa.eu/smart-regulation/impact/ia_carried_out/docs/ia_2004/sec_2004_0931_en.pdf .

(11)     https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/cmef/rural-areas/synthesis-ex-ante-evaluations-rural-development-programmes-2007-13_en .

(12)    See footnote 6.

(13)    See footnote 6.

(14)    The contractor used 91 rural development programme ex post evaluation reports.

(15)     NAT/699-EESC-2017-00690-00-00-RI-TRA .

(16)    European Evaluation Network for Rural development, “Capturing the success of your RDP: Guidelines for the ex post evaluation of 2007-2013 Rural development programmes”, Brussels, 2014.

(17)    A contractor who provided evaluation-related technical and administrative assistance.

(18)    Final score (E); Number of all reports (R); Number of positive results (P); Number of negative results (N); Number of limited results (L); Constant factor of 0.5 for weighting of limited results (C).

(19)    See footnote 14.

(20)    Poland (Warsaw), Italy (Florence), Finland (Helsinki/Turku) and Spain (Seville).

(21)    Austria, Belgium, Bulgaria, Finland, France, Germany, Ireland, Italy, Luxembourg, the Netherlands, Poland, Spain and the UK.

(22)    The assessment of the mid-term evaluation reports in 2011 revealed the need to enhance the quality of evaluation since frequently these reports did not succeed in adequately demonstrating the achievements, results and impacts of the rural development programmes. Therefore it was suggested to revise and reduce the number of the current set of evaluation questions outlined in the CMEF.

(23)     Guidelines for the ex post evaluation of 2007-2013 Rural Development Programmes, pages 29 and 30 .

(24)     https://ec.europa.eu/agriculture/sites/agriculture/files/statistics/rural-development/2013/full-text_en.pdf .

(25)     See DG .

(26)    See DG AGRI data dashboards: CTX_SEC_14_ 1.

(27)    See DG AGRI data dashboards: CTX_SEC_17_ 1.

(28)    See DG AGRI data dashboards: CTX_SEC_17_ 2.

(29)    See DG AGRI data dashboards: CTX_SEC_25_ 1.

(30)    Note: total factor productivity (TFP) compares total outputs relative to the total inputs used in production of the output. As both output and inputs are expressed in term of volume indices, the indicator measures TFP growth. The TFP a composite indicator for land, capital and labour productivity growth. The comparison between Member States of the change over time of TFP growth is meaningful but not the comparison of the indicator as such. This is an index, in the first step 2010 is set at 100, then 3 year-averages are calculated to smooth the effect of weather e.g. on the indicator. Therefore, in the graph 2010 corresponds to the (2008-2010) average.

(31)    Sum of ‘strongly positive’ and ‘moderately positive’ views.

(32)    It also should be noted that 31% of the respondents provided no opinion on the increase of competitiveness in forestry (compared to 10% of competitiveness in agriculture). If the analysis is made only of those that provided an opinion, the positive view increases to 64% of the respondents, with no significant differences between stakeholder groups.

(33)    This is an over-estimate, given that several operations might have taken place on the same holding.

(34)    76 of the ex-post evaluations reported on the contribution of the measure.

(35)    70 of the ex-post evaluations provided a conclusion on M123.

(36)    59 of the ex-post evaluations provided a conclusion on M125.

(37)    Only 40 of the ex-post evaluation reports provided a conclusion on M113.

(38)     Evaluation staff working document on the impact of the CAP on generational renewal .

(39)    See DG AGRI data dashboards: CTX_SEC_23_5 .

(40)     https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=41529 .

(41)    78% (59) of those ex-post evaluation reports that provided a conclusion on M112.

(42)    71% (58) of those ex-post evaluation reports that provided a conclusion on M111.

(43)    See Eurostat (online data table aact_ali01).

(44)    The decrease in the EU-N13 was 3% per year in 2007-2016 and in the EU-15 it was 2% per year for the same period.

(45)    See DG AGRI data dashboards: CTX_SEC_13_2 b.

(46)    See footnote 37.

(47)     CAP performance: 2014-20 | European Commission (europa.eu) .

(48)     http://pubdocs.worldbank.org/en/369851513586667729/Thinking-CAP-World-Bank-Report-on-the-EU.pdf .

(49)    Dumangane, M., Freo, M., Granato, S., Lapatinas, A. and Mazzarella, G., An Evaluation of the CAP impact: a discrete policy mix analysis, Publications Office of the European Union, Luxembourg, 2020, JRC125451.

(50)    See footnote 14.

(51)    Sum of ‘moderately’ and ‘strongly’ positive opinion.

(52)    See DG Agriculture and rural development data dashboards: CTX_ENV_31 .

(53)     Staff Working Document Evaluation Impact of the CAP on biodiversity, soil and water .

(54)    See DG Agriculture and rural development data dashboards: CTX_ENV_33_2 .

(55) The link between land abandonment and biodiversity loss (among other issues) is demonstrated in literature, i.e.: Assessing the risk of farmland abandonment in the EU

(56)    See DG Agriculture and rural development data dashboards: CTX_SEC_21_1 .

(57)    73% (52) of those ex-post evaluation reports that provided a conclusion on M211 and M212.

(58)    40 (53.4%) of the ex-post reports provided positive conclusions on M225.

(59)    See DG Agriculture and rural development data dashboards: CTX_SEC_29_1 .

(60)    62 (72.1%) of the ex-post reports provided positive conclusions on M226.

(61)    See DG Agriculture and rural development data dashboards: IMP_07_2 and IMP_07_1 .

(62)    See agridata: CTX_ENV_43_1.

(63)    CTX_ENV_45_1a.

(64)    See DG Agriculture and rural development data dashboards: CTX_SEC_20_01 .

(65)    See DG Agriculture and rural development data dashboards: IMP_11 .

(66)    Report on the implementation of Council Directive 91/676/EEC for the period 2.008–2011. (COM (2013) 683 final).

(67)    From 91 evaluations, 13% of the reports reported on water usage and only one refers to water quantity.

(68)    See DG AGRI data dashboards:  CTX_SOC_10_2a .

(69)    See DG Agriculture and rural development data dashboards: CTX_SOC_2 .

(70)    See DG Agriculture and rural development data dashboards: CTX_SOC_11_6, CTX_SOC_11_5; CTX_SOC_11_6 .

(71)    Another 28% of the reports did not provide data on number of jobs created as it was not a programme priority.

(72)    Technologies needed to meet the Digital Agenda of the 30 Mbps objective.

(73)    Digital Economy and Society Index Report 2018 - Use of Internet Services; https://ec.europa.eu/digital-single-market/en/use-internet .

(74)     https://www.eca.europa.eu/Lists/ECADocuments/SR13_12/SR13_12_EN.pdf .

(75)     https://www.eca.europa.eu/Lists/ECADocuments/SR15_20/SR15_20_AGRI_INVEST_EN.pdf .

(76)     https://www.eca.europa.eu/Lists/ECADocuments/SR15_12/SR_RURAL_TRAINING_EN.pdf .

(77)     https://www.eca.europa.eu/Lists/ECADocuments/SR14_22/SR14_22_EN.pdf .

(78)    See footnote 14.

(79)    Brandenburg/Berlin (Germany), Castilla La Mancha (Spain), England (United Kingdom), Hamburg (Germany) and Latvia.

(80)     See synthesis of ex-ante evaluations of rural development programmes 2007-2013 .

(81)    See footnote 14.

(82)    See footnote 14.

(83)    See footnote 74.

(84)    European Commission, A long-term Vision for the EU’s Rural Areas, COM (2021)345 final, EUR-Lex - 52021DC0345 - EN - EUR-Lex (europa.eu) .

(85)    For the purpose of this evaluation, risk of poverty or social inclusion is defined as ‘the sum of persons who are either at risk of poverty, or severely materially and socially deprived or living in a household with a very low work intensity’ .

(86)    This is different from the CMEF that covered the whole of the CAP in the 2014-2020 programming period.

(87)    Council Regulation (EC) 1305/2013 and Regulation (EC) 808/2014.

(88)    See footnote 14.

(89)    See footnote 14.