1.4.2008   

EN

Official Journal of the European Union

C 81/1

I-VII

1-8

1-7

8

9-13

14-34

14-16

17-20

21-25

26-34

35-55

35-39

40-41

42-49

43-46

47-49

50-55

56-58

59-65

59-62

63-65

Cohesion Fund

Instrument designed to promote economic and social cohesion by financing large projects in the fields of the environment and transport in Member States with a per capita GNP of less than 90 % of the Community average. The Cohesion Fund was originally implemented in Spain, Greece, Ireland and Portugal. Since 1 January 2004 Ireland has no longer been eligible. The new Member States have profited from Cohesion Fund support since their accession to the Community.

Cost-benefit analysis

A technique for comparing all the costs and all the benefits of an intervention to determine whether the benefits outweigh the costs, and if so, by what proportion.

Deadweight effect

An effect that becomes apparent and would have emerged even if a public intervention had not taken place.

Effectiveness

Measures the relationship between the results obtained and the objectives set.

Efficiency

Measures the relationship between the resources employed and the results obtained.

ERDF (European Regional Development Fund)

Financial instrument designed to promote economic and social cohesion between the countries of the EU by correcting the main regional imbalances and by reconversion of regions whose development is lagging behind. ERDF interventions are mainly implemented through operational programmes encompassing a large number of projects.

Impact

Socioeconomic consequences that may be attributed to an intervention.

IRR (internal rate of return)

Rate at which an investment becomes profitable.

NPV (net present value)

Value obtained by indexing at the present time the value of all cash balances which will be generated by a project during its life. This method enables various projects to be compared.

Operational programmes

A document approved by the Commission, it takes the form of a coherent set of priorities comprising multiannual measures. The priorities may be implemented through recourse to one or more Structural Funds, or, alternatively, to other financial instruments and the EIB.

Programming period

The multiannual framework within which Structural Fund expenditure is planned and implemented.

Risk analysis

Study to determine the probability that the value of the underlying variables originally calculated for a project will actually be confirmed.

Sensitivity analysis

Study of the stability of the expected result of a project when exposed to changes in the underlying variables.

Structural Funds

Community funds financing actions with the aim of reducing disparities in development between the regions of the EU. ERDF is the main financial instrument for these actions.

I.

The regulations define ‘major’ infrastructure and productive investment projects essentially in terms of the volume of finance involved. In view of the economic significance and potential impact of these projects, they are subject to special preliminary examination procedures prior to approval. For the same reasons, relevant information must be collected so that the effects actually achieved can be assessed in comparison to those which had been expected (see paragraphs 1-8).

II.

The Court's audit centred on ERDF major projects and, by analogy, those under the Cohesion Fund for the 1994-1999 and 2000-2006 programming periods. This report is constructed around two main themes: the process of preparation for approval, by the Commission, of major projects and the process of evaluation after their implementation (see paragraphs 9-13).

III.

The Court found several weaknesses in the preliminary examination procedure in the two periods in question. Of particular concern were the procedures for project appraisal and for setting the rate of Community co-financing. The quality of the cost-benefit analyses presented by project promoters in support of applications for assistance was extremely variable, and the same was true of the use to which the Commission put those analyses. These findings lead to doubts about the real value added of the special procedure for approving major projects (see paragraphs 14-34).

IV.

The Court recommends that the Commission should apply more efficient preliminary examination procedures based on the tools it has itself developed, should ensure that the cost-benefit analysis is rigorously applied, particularly in respect of the data provided, and should continue its efforts to take into account those factors which allow the rate of Community participation to be adjusted (see paragraph 63).

V.

As regards the ex post evaluation of major projects from the 1994-1999 period (the only such evaluation that the Commission had been able to carry out at the time of the audit), it proved to be severely handicapped by the lack of pertinent and accurate information on anticipated effects and monitoring. Given these circumstances, the evaluations did not produce any information that was useful from the point of view of decision-making for the following period, with the exception of one case concerning projects cofinanced by the Cohesion Fund. The evaluation of ERDF major projects from the 2000-2006 period, which has yet to be carried out, will be able to rely on stricter rules, especially in the area of monitoring. However, not much has yet been done to apply those rules in practice (see paragraphs 35-55).

VI.

The Structural Funds regulations for the 2007-2013 period do not require the Commission to carry out individual ex post evaluations of major projects. However, the Court recommends that, when completed, the major projects should be evaluated individually, in relation to their specific objectives, by the national management authorities and the beneficiaries, for whom the information obtained from this exercise would be useful. To this end, the Commission must ensure that the applications for assistance contain the information required in respect of indicators and the initial situation, and that this information is rigorously monitored. As regards the information provided by the evaluation, which must be forwarded to it, the Commission must analyse its own requirements so as to ensure that it has relevant, reliable data available on the effects produced by the major projects (see paragraph 64).

VII.

In the longer term, the Court invites the Commission, the European Parliament and the Council to give thought to the purpose of the special arrangements for major projects (see paragraph 65).

1.

The concept of ‘major projects’ is defined, in essence, in the Community regulations on the ERDF (1), and depends on the volume of finance involved in the projects.

2.

For the 1994-1999 programming period, ‘major projects’ were ‘those the total cost of which taken into account in determining the amount of Community assistance is, as a general rule, greater than ECU 25 million for infrastructure investments or greater than ECU 15 million for productive investments’ (2).

3.

For the 2000-2006 programming period, the definition of ‘major projects’ covers projects ‘whose total cost taken into account in determining the contribution of the Funds exceeds 50 million euro’ (3). In addition, these projects must comprise an economically indivisible series of works fulfilling a precise technical function and which have clearly identified aims. As in the previous period, these may be investments in infrastructure or productive investments (4).

4.

The Cohesion Fund regulations, on the other hand, contain no definition of ‘major projects’ for either one of the periods under consideration (5). However, since, in the cohesion countries, similar transport infrastructure or environment projects may be financed by either the Cohesion Fund or the ERDF, the audit also examined, by analogy, a number of Cohesion Fund projects that were selected on the basis of total cost, applying the same criteria as for the ERDF major projects.

5.

As regards the approval framework, ERDF major projects for the 1994-1999 period could be submitted individually or within the framework of an operational programme. In any case, the Commission was informed about major projects, with further details to be supplied (6), but no formal approval decision was required.

6.

All ERDF major projects for 2000-2006 were financed within a wider programme of assistance (7). However, the regulations required the Commission to approve them individually, unlike other projects, which were approved only at Member State level (8).

7.

Under the Cohesion Fund, in both periods, only individual projects could be financed, each project forming the subject of a specific decision.

8.

Table 2 shows the total cost for the two programming periods of the major projects which received assistance from the EDRF and the Cohesion Fund:

9.

Given the impact that single projects of a financial volume exceeding a certain amount may have on regional development, major investment projects are bound by preliminary examination procedures that are more than usually complex. As of the 2000-2006 period, at least, they are also subject to a separate approval decision. For the same reasons, a retrospective appraisal of the actual effects generated by the allocated resources is required in the interest of obtaining guidance for project design and selection in future programming periods.

10.

The Court therefore sought replies to the following questions:

11.

The administrative checks and checks carried out in the Member States covered a sample of 20 closed major projects from the 1994-1999 period, and 30 major projects from the 2000-2006 period. In addition, a supplementary sample of 36 major projects from 2000-2006 was taken for administrative checks at the Commission (see Annex I).

12.

With regard to the system that was set up for the preliminary examination of major projects, the audit focused on the Commission's procedures for the two periods, 1994-1999 and 2000-2006. The progressive improvements made are indicated. The recommendations in this part will probably have an impact on implementation in the 2007-2013 period, for which the concept of ‘major projects’ has been retained for all the Structural Funds (9).

13.

Turning to the subject of ex post evaluation, only those evaluations concerning the projects of the 1994-1999 period could be taken into account, since, at the time of the Court's audit, implementation of the projects of the 2000-2006 period had not yet been completed and the ex post evaluations were still to be done (10). Nevertheless, the report also examines whether improvements can be hoped for in respect of the 2000-2006 period. The observations that were drawn up on conclusion of the audit may have an impact on the evaluations for the 2000-2006 period, which are yet to be carried out, but chiefly on those for the 2007-2013 period which has just begun.

14.

It is the responsibility of the Member States to present major projects for Community co-financing, and this is carried out on the basis of their own preliminary examination procedures and rules, taking into account the stated priorities of the relevant ERDF Operational Programmes and of the Cohesion Fund.

15.

The Commission is not expected to make a comparative assessment of the different Member States' projects, as the volumes of finance earmarked for each State are fixed in advance during the preparations for each programming period. Nevertheless, the Commission must examine the quality of the projects to be financed, i.e. whether the projects selected have the potential to achieve results with regard to regional development, and whether financial participation by the Community is required to carry them out and would deliver real added value.

16.

The Court scrutinised the Commission's preliminary examination phase, on the basis that the following factors were necessary for this phase to lead to the adoption of worthwhile projects:

17.

The Court's audit demonstrated that, with time, the management tools put in place by the Commission have been refined. Although there are differences between the application forms for assistance for ERDF and Cohesion Fund major projects for the 2000-2006 period, they are more comprehensive and suitable than the forms used in the preceding period. Likewise, the Guide to the cost-benefit analysis (11) made available to the Member States by the Commission has been improved both in terms of its presentation of the theoretical framework and its practical application (12).

18.

However, the examination of the applications varied in quality:

Box 1With regard to productive projects, one Commission department had pointed out that the investments were likely to prove extremely profitable and therefore did not require assistance. For one transport infrastructure project, another department had proposed that the requested 60 % contribution rate be reduced because the project was located in the wealthiest region of the Member State and the authorities were applying lower rates in regions that were less well-off. A rate cut was also sought in the case of another project where the investment return was extremely good. In the end, the reduction in the requested contribution rates was not made for any of these projects. In the case of the construction of an underground railway, it was converted into an environmental project under the Cohesion Fund when DG TREN refused to view it as transport infrastructure, despite reservations expressed by DG ENV on the grounds that only 1-3 % of the benefits to be derived from the project would be environmental in nature.

Box 2In Greece, assignment to one or the other fund, following consultation with the Commission, depends on the projects' level of preparation and planning and the availability of resources. This has led to the somewhat illogical situation where one extension project for a major transport project was financed by the Cohesion Fund at the rate of 66,3 % and another by the ERDF under an operational programme at 50 %.

19.

There were substantial delays in the examination process. The two-month time limit set for Commission approval (13) following the receipt of application forms is usually exceeded. The average time needed for the ERDF and Cohesion Fund projects that were audited on the spot was 187 days, with the approval period ranging from 46 to 1 001 days. These delays are attributable to the fact that the Commission is obliged to seek further details on the vast majority of major projects, largely owing to the poor quality — or absence — of environmental impact studies, which are compulsory in many cases, and to specific aspects of the financial and economic project analyses (14). Furthermore, the Member States are often very slow in replying. The shortcomings which affected project management for a considerable time may, in part, explain these delays (see paragraph 17).

20.

The Commission's examination for the approval of projects proved to be too lenient:

21.

The cost-benefit analysis required by the legislation, in order to substantiate applications for assistance, is the main tool at the Commission's disposal for its decision to grant Community financing or not. Its satisfactory application is essential to evaluate the quality of the projects submitted. Its objective is to compare the resources deployed with the socioeconomic benefits which may be obtained in the medium term from the proposed action (16). The Commission may ask for the analysis submitted in support of the application to be revised or expanded, or may itself carry out an appraisal of the project, seeking the views of independent experts, where necessary, and the assistance of the EIB.

22.

Despite publication by the Commission of a Guide to the cost-benefit analysis of major projects (see paragraph 17), great disparity was noted in the quality of the financial and economic analyses presented by the promoters.

23.

The Commission takes an equivocal attitude to this. In some cases it tolerates numerous weaknesses and approves projects without requiring further details, while in other cases it suspends approval until it has received essential items of financial information.

24.

The Court's audit has brought to light the following shortcomings and inconsistencies which impair the appraisal of the projects:

Box 3In the case of the construction of a cross-border tunnel, the application, as a major project, was submitted by one of the two Member States involved. An overview of the entire tunnel project would have been useful for an evaluation of the applicant Member State's section. The same applies to the cross-border motorways. For the national motorways and railways a separate economic analysis was made in many cases for each phase, with each phase constituting a major project, but there was no overall economic analysis.

Generally speaking, the quality of the analyses is weaker in the case of major productive investment projects.

25.

The poor quality of the analyses accompanying the aid applications shows that, in the examination phase, the Commission did not always carry out an effective examination of the cost-benefit analyses of the projects. Whether infrastructure investments or productive investments, under these circumstances conclusive rejection of a project was a very rare occurrence, as rare as requests for projects to be revised after examination by the Commission.

26.

The purpose of the administrative examination undertaken by the Commission is to ‘confirm or amend the rate of Community assistance’ (17). Introduced to ensure that awards are effective, this instrument ensures that the financed project receives the resources needed for its achievement without the beneficiary acquiring any unjustified advantage. This is particularly important where the beneficiaries are private businesses, as Community funding may distort market conditions.

27.

Community assistance is subject to certain regulatory ceilings. Lower regulatory ceilings are provided for investments in infrastructure that generate substantial net revenue and for investments in firms (18). In addition, Community rules and State aid ceilings must be complied with.

28.

Within the limits of these ceilings the only precise factor used by the Commission to determine the rate of Community assistance has been the financing gap ratio, which is only applied to revenue-generating infrastructure projects. It consists of calculating that part of the discounted initial investment cost of the project which is not covered by the net present value of operating revenues (including the residual value) (19). This calculation allows the maximum level of public intervention to be fixed. In 2003 the Commission introduced an amended method of calculation for major sanitation and solid waste management projects (20), which takes account of revenue generation while seeking to avoid penalising beneficiaries who rigorously observe the ‘polluter pays’ principle (21).

29.

In practice, in certain cases examined by the Court, the application of the financing gap ratio principle has either resulted in fixing too high a rate of Community contribution or there is doubt as to the correct application of this principle.

30.

Apart from the financing gap method, no guidelines indicate any other factors to be taken into account for the purpose of adjusting Community assistance rates, which helps to explain certain differences in the views expressed during the interdepartmental consultation process (see paragraph 18). In the absence of previously established criteria, the rate of assistance is adjusted in a random manner and its consistent application cannot be verified.

31.

The Court's audit has shown that as regards public infrastructure investment projects which do not generate revenue, socioeconomic aspects (‘context’ factors such as the degree of industrialisation of a region or its relative prosperity) are not taken into account for the purpose of reducing assistance rates. The same applies to such criteria as complementarity with other Community operations. The Commission also did not question the structure of the financial arrangement submitted by the project initiator.

32.

As regards productive investments, the review of the projects has shown the method of setting assistance rates to be relatively automatic:

Box 4In the case of one company generating substantial profits, the project would have succeeded without Community assistance. The same applies to another industrial production project, for which the business concerned had almost no competitors in the European market. However, in certain cases the effects of locating a project at a particular site might justify subsidies for projects with favourable IRRs. In the case of a major project of another company, which, at the time, had substantial shares of the world market for several projects, the cofinanced investment was at a company site in an Objective 1 region and Community grants played a significant role in the choice of site.

33.

The Commission is aware of these weaknesses. In 2004 it made an attempt at rationalisation by seeking to adjust Community assistance rates by reference to certain criteria. This produced indicative assistance rates weighted by each sector's anticipated rate of return, calculated on the basis of financial IRRs from a sample of 200 infrastructure and productive projects. This is a useful tool, although still very limited. In any case, it came too late for examination of the files.

34.

The new guidelines published in 2006 (22) propose including in the financing gap calculation factors that are not purely financial, such as the existence of a public-private partnership and equity (that is, the relative prosperity of the region). They also deal with the manner of combining subsidies and loans on the basis of the profitability normally expected per sector. Even though ‘context’ factors are still not fully taken into account, these improvements are in line with the approach favoured by this report of seeking a wider and more objective framework for differentiating the amount of aid. They must be applied consistently and transparently to the financing decisions of the new period.

35.

Generally speaking, the aim of evaluating a project after completion is to account for the use of resources and the effectiveness and efficiency of the operation undertaken. It is of particular interest to political leaders, managing authorities and beneficiaries. It must allow lessons to be drawn about the need for supplementary or corrective measures to be taken in the future. As the results generated by such evaluations concern the effects produced by a project, they will also be used in quantifying the objectives of future projects.

36.

The Structural Funds regulations stipulate that Community interventions shall be subject to such a type of evaluation. It is characterised as an ‘ex post evaluation’ and must be undertaken by the Commission in collaboration with the Member States (23). As an ERDF major project that is submitted outside an operational programme is regarded as an intervention, it is in principle subject to ex post evaluation. There is no explicit regulatory requirement as regards the individual major projects that make up an intervention (24).

37.

However, as regards the Cohesion Fund, there is express provision for ex post evaluation of each project (25).

38.

The basic data used in the ex post evaluations are mainly collected by the Member States through the monitoring system. During the 1994-1999 period, only Cohesion Fund projects were subject to individual monitoring and annual implementation reports. Structural Funds projects were explicitly monitored at a higher level (measures, priorities, operational programmes, Community support frameworks, etc. (26)).

39.

The Court examined the evaluation phase from the point of view that:

40.

The Member States only rarely carry out ex post evaluations of major projects (27). Only one of the twenty projects audited by the Court had been evaluated (see Box 5). Naturally, studies of the sectors to which the major projects belong — such as, for example, the port and automotive sectors — may be conducted by beneficiaries, managing administrations or external bodies (universities or research centres). However, such studies do not target the projects directly and are not in the nature of ex post evaluations. Furthermore, they are not always forwarded to the departments responsible for managing the Community funds.

Box 5Only one of the projects in the Court's sample had undergone individual ex post evaluation. However, the macroeconomic impact was assessed on the basis of factors which differed from those initially provided for in the ex ante evaluation presented in support of the application. As a result, the impact cannot be compared with the one anticipated. What is more, there was no recalculation of the expected financial return.

41.

At Commission level, the evaluations that specifically target the major projects of the 1994-1999 period (the only ones to have been evaluated up to now) (see paragraph 13), or concern some aspect of them, have very heterogeneous aims and methods. They were not used by the national managing authorities and beneficiaries, with one exception (see (d)):

42.

The Court has on several occasions (30) highlighted the weaknesses of the evaluation exercises for the 1994-1999 period, which were particularly marked by the absence of data or its unreliability. The Court's reports invited the Commission to consider the possibility of collecting information on the individual projects, without which the methodologies used are too theoretical or inapplicable. The audit of the major projects of that period corroborated these findings.

43.

The ex post evaluation of major projects was at first hindered by missing information in the aid applications. This was noted both by the studies undertaken of major projects and by the Court's audit.

44.

One of the evaluations carried out highlighted, in particular, that:

45.

In addition to the inadequacy of the cost-benefit analysis that has already been highlighted (see paragraphs 21-25), the audit carried out by the Court confirmed the shortcomings found in the earlier reports, in particular:

46.

As these factors are necessary in order to evaluate the operations, the deficiencies discovered impede ex post evaluations.

47.

Even in those cases where a monitoring system was introduced at project level, the studies undertaken and the audit carried out by the Court have shown that the monitoring process did not ensure that adequate data was collected for the purposes of evaluation.

48.

One of the evaluations carried out found that:

49.

The Court's audit of major projects for the 1994-1999 period also showed weaknesses in monitoring:

50.

For the 2000-2006 period, the ERDF legislation stipulates that, in future, for the major projects, the annual implementation reports on interventions must contain a separate chapter on progress made and financing. In addition, financial and progress indicators must be designed to provide separate information for the major projects (31).

51.

Nevertheless, although the annual implementation reports contain a chapter on major projects, these chapters are often restricted to a simple description of the projects and do not present the physical progress made, or separate results and impact indicators.

52.

In the same way, the priorities and measures concerning major projects do not always include the indicators allocated to them, or the indicators at priority and measure level are not consistent with those for the major projects.

53.

At the level of the projects examined on-the-spot, while a number of major projects have an adequate list of implementation, results and impact indicators, other projects either do not refer to any indicator at all, or have only been allocated physical and financial implementation indicators.

54.

Sometimes no clear link exists between the overall objectives, the individual objectives and the designated indicators. A number of objectives have not been allocated indicators at all. Likewise, indicators put in place for projects which are similar but cofinanced by the ERDF or the Cohesion Fund respectively are not identical.

55.

Future ex post evaluations will be affected for the same reasons as in the preceding period, i.e. the weaknesses found concerning monitoring and indicators. The contribution of major projects to the more global objectives of regional development will therefore remain very difficult to define.

56.

The new framework regulation for the 2007-2013 period (32), which applies both to the ERDF and the Cohesion Fund, introduces certain improvements as regards major projects:

57.

In addition, in 2006 the Commission published guidelines on the preparation of major projects in the new period, which made a certain number of improvements (see paragraph 34).

58.

Finally, the rules provide for ex post evaluations to be carried out by the Commission at programme level only. There is therefore no provision for either EDRF or Cohesion Fund projects to be evaluated individually.

59.

There are a number of shortcomings in the preliminary examination procedures for major projects:

60.

Taking these factors into consideration, the Court entertains doubts concerning the real added value of the approval process for major projects as implemented by the Commission. The exercise is cumbersome and does not exclude projects of inferior quality or adjust the rate of intervention in a satisfactory manner. Neither does it guarantee adequate preparation for the subsequent evaluation. The weaknesses noted in the preliminary examination procedure had repercussions on the ex post evaluation process.

61.

Weaknesses were also discovered in the ex post evaluations:

62.

The Commission adopted a number of measures in 2003 in the light of the weaknesses found at the end of the ex post evaluation of the 1994-1999 period. However, no substantial progress was made with the adoption and monitoring procedures for the major projects of the 2000-2006 period and the ex post evaluation exercise for this period, which is still to be carried out, is at risk of being compromised (see paragraphs 50-55).

63.

As regards the 2007-2013 period, the Community regulations provide that, as in the preceding periods, the Commission should approve the granting of Community assistance for each major project. To avoid financing decisions being reduced to an administrative exercise with no influence on the quality of the projects adopted, the Commission should:

64.

The Structural Funds regulations for the 2007-2013 period do not provide for the ex post evaluation of major projects by the Commission. However, the Court considers that the evaluation of major projects after completion is essential for the national administrations and the beneficiaries, who need to verify the real efficiency and effectiveness of the expenditure in order to improve management and for whom the evaluation should provide real assistance with regard to programming. As a result:

65.

In the longer term, the Court invites the Commission, the European Parliament and the Council to give thought to the purpose of the special arrangements for authorising major projects.