52002DC0331

COMMUNICATION FROM THE COMMISSION - A Methodological Note for the Horizontal Evaluation of Services of General Economic Interest

1. Introduction: background and justification.

In its report to the Laeken European Council of October 2001 [1], the Commission defined a strategy for the efficient evaluation of the performance of services of general interest based on:

[1] COM (2001) 598 of 17.10.2001

i) The reinforcement of sectoral reporting.

ii) The annual horizontal evaluation of these services in the framework of the Cardiff process and,

iii) The benchmarking of the effectiveness of measures taken in the Member States to attain adequate performance of services of general interest in areas not covered by sectoral reporting or the annual horizontal evaluation. For this, the Commission would provide assistance to Member States for benchmarking purposes.

The strategy proposed here is consistent with the Commission's general initiatives to improve governance and the quality of regulation in the European Union. [2]

[2] European Governance, A White Paper , COM(2001) 428. 25.7.2001; Action Plan "Simplifying and improving the regulatory environment", COM(2002) 278, 5.6.2002.

On 7 December 2001, the Commission presented "Market performance of network industries providing services of general interest: a first horizontal assessment" [3] annexed to the "Report on the functioning of product and capital markets [4]". This was a first response to the European Council's request made in the declaration on services of general interest attached to the Nice European Council conclusions. It provided a baseline for the future horizontal monitoring and evaluation of these services.

[3] SEC (2001) 1998

[4] COM (2001) 736 final

The Laeken European Council, the Internal Market, Consumer affairs and Tourism Council and the Barcelona European Council, as well as the European Parliament, have all invited the Commission to present a methodology for the evaluation of services of general interest and to carry out regular evaluations of these services. This communication responds to those invitations by defining a methodology to be applied by the Commission in future horizontal evaluations.

Horizontal evaluation does not preclude or interfere in any way with the sectoral monitoring of markets or network industries. On the contrary, by providing a horizontal assessment of the results and impacts of national and Community policies, it will enrich and supplement the information obtained through sectoral monitoring.

Evaluating network industries providing services of general interest at this particular time is justified by the fact that these sectors are currently undergoing important structural reforms due to regulatory technological, social and economic changes. Furthermore, network industries providing these services have a significant impact on competitiveness as they account for a large part of EU GDP and employment and the services they offer represent a sizeable input for the European economy.

There is a greater need for industries providing services of general interest to be the subject of horizontal evaluation than for other services having also a key impact on competitiveness, because these services meet important social and public policy objectives. In particular, these services can contribute to strengthen social and geographic cohesion, because market outcomes do not or may not provide the socially desirable level of service provision. It is thus necessary to evaluate the performance of these sectors, to ensure that the current structural changes do not prevent those social and public policy objectives being attained.

If we are to achieve the Treaty's objectives of sustainable growth, employment and cohesion , it is essential that services of general interest perform well. Horizontal evaluations must thus be carried out at Community level. As explained below, the subsidiarity principle must be applied when defining the scope of these evaluations.

Regarding the benchmarking of measures mentioned in point (iii) above, taken in order to attain adequate performance of services of general interest and not covered by these horizontal evaluations nor by sectoral reports, the Commission intends to examine possible requests by Member States on a case-by-case basis.

2. Objectives

This Communication develops the Commission approach laid out last year. It seeks to make a positive contribution to the decision making process in all policy areas related to network industries providing services of general interest by providing a methodology that can help produce precise, thorough and comparative evaluations of the performance of those network industries and the services they provide. Rigorous and regular assessments of performance will provide guidance for policy making, especially in the context of the market opening and reform process, and help determine whether market outcomes are compatible with the social and economic objectives of the European Union.

The evaluations will be based on factual information on the evolution of those industries and feedback from citizens, consumers and other stakeholders. They will monitor and report on the performance of network industries providing services of general interest as they are currently impacted by structural reforms and the process of opening up to competition. In this respect, the horizontal evaluations will focus on the following questions:

1. How has competition evolved in these industries and, in particular, how important was the influence of the market opening process on these changes-

Examining the evolution of competition in these sectors, especially as a result of market opening, will take account of changes in the structure of both supply and changes demand. The former include changes in the number of competitors, the evolution of prices, the degree of openness of the market in legal terms and the evolution of concentration. The latter will involve, for example, investigating whether more freedom of choice has led to users switching suppliers.

2. Have these changes had an impact on market performance-

A second objective is to assess whether changes in competition have had a positive impact on the performance of the network industries in terms of productivity, innovation, employment, or growth, but also on the equally important aspects of affordability, quality and accessibility of services of general interest.

3. How have these changes been perceived by users-

As suggested by the September 2000 communication [COM(2000) 580] and the 2001 Commission report [COM(2001) 598], attention should be paid to consumers' perceptions of market performance [5]. To assess whether this changing performance is transparent to and valued by consumers, the Commission will undertake consumers' satisfaction surveys. More generally, the perception of all stakeholders needs to be taken into account. Any discrepancy between users' perception and actual market performances by these industries will indicate the need for more transparency, better communication, and better dialogue with users about their performance.

[5] The term "users" includes both business and household consumers. For simplicity, the latter will be refered to as consumers hereafter.

3. Proposed approach

3.1. Principles

The following principles have been taken into account by the Commission in preparing this methodology.

a) The methodology has to be adapted to the evolutionary nature of services of general interest.

As stated by the Commission in the 2000 Communication on services of general interest, these services are subject to constant social, economic and technological evolution. It is not therefore possible or useful to define ex-ante a precise and permanent structure for their horizontal evaluation. The evaluation's sectoral scope and thematic focus must evolve with the services themselves and with data availability.

b) The methodology needs to be comprehensive, taking into account economic, environmental and social dimensions of the market performance of network industries providing services of general interest.

These services have an impact on economic development, job creation, social cohesion, the balanced and harmonious development of regions, sustainable development and other important social objectives such as safety or stability in the provision of the services. The methodology should ultimately cover all these dimensions and insofar as data availability permits, it will do so from the outset.

However, as the first horizontal evaluation showed, this is far from being possible at this stage and additional efforts are needed to fill gaps in current data availability by developing new indicators, especially to measure the qualitative dimensions of market performance and the externalities resulting from the market outcomes of these service sectors. Implementation of this methodology and co-operation with stakeholders should allow us to expand the range of indicators available in the longer term.

c) The methodology must be based on the full application of the subsidiarity principle.

As stated in the 2001 report on services of general interest, "it is generally up to the Member States to define specific performance requirements for these services and to ensure that these requirements are complied with". This methodology thus focuses on the Community dimensions of the performance of services of general interest, in particular as regards the achievement of the public policy objectives defined in the regulatory framework established at Community level for the provision of these services. The focus will be on the impact on integration and on the functioning of the Internal Market taking into account both the business and consumer perspectives. The analysis will take into account the situation in different Member States, but the yardstick for evaluation will be the objectives established at EU level. The analysis will nevertheless also seek to highlight value added and identify "best practices" in Member States.

Successful implementation of this methodology relies heavily on co-operation with Member States' authorities and national regulators, whose contribution and co-operation will be essential if we are to succeed in improving the information available. However no new process nor reports will be imposed on Member States to implement this methodology.

d) The methodology must be transparent and pluralist given the clear social dimension of these services.

This methodology has to take into account the opinion of citizens, as consumers and stakeholders, as an essential element in the evaluation of market performance. As announced in last year's report, "a permanent mechanism for the monitoring of citizens' opinion and their evolution" will be established. Stakeholders, including the social partners, will also be consulted on an ad-hoc basis for specific issues.

Public participation could be greatly expanded, as suggested in the European Parliament's resolution [6]. The Parliament proposes to "organise the debate within the various existing forums (Economic and Social Committee, Committee of the Regions, consultative bodies, associations involved in services of general interest initiatives and consumer associations)". The results of this debate should be taken into account and provide guidance for the annual horizontal evaluation, and the evaluation should itself be the subject of debate.

[6] European Parliament report on the Commission "Communication on services of general interest in Europe" COM(2000) 580 C5-0399/2001 - 2001/2157/COS; Final A5/0361/2001. Rapporteur, Werner Langen. 17 October 2001.

3.2. Scope

The evaluations will focus on services of general economic interest and non-economic services of general interest [7] will not be subject to evaluation. Initially, horizontal evaluations will cover the following sectors: Air transport; local and regional public transport; electricity; gas; postal services, railway transport and telecommunications.

[7] See appendix to COM(2001) 598 of 17.10.2001 for definitions of services of general economic interest and non-economic services of general economic interest. Services of general economic interest is the term used in Art 86 of the Treaty and refers to market services which the Member states subject to specific public service obligations by virtue of a general interest criterion.

This sectoral coverage could be expanded (or even reduced) in the future, depending on the evolution of the different sectors. Broadcasting would be among the possible candidates. In those cases, the methodology will evolve to meet the specific features of new sectors to be covered. Evaluations will in principle cover all Member States (subject to data availability for specific topics). When appropriate and subject to data availability, attention will be paid to regional information within Member States. New Member States will be covered starting in 2005.

4. Output and structure of reports

The Commission will produce annual reports presenting the results of the horizontal evaluation of services of general interest in the context of the Cardiff process on economic reform. The timing of these reports will be adapted to the timetable for the production of the Commission's report contributing to the Special Spring European Councils. They will consist of three main parts.

4.1. Analysis of structural changes and market performance.

In this section, the Commission will use quantitative and qualitative indicators to assess developments in the competitive framework and the resulting market performances of network industries providing services of general interest. Insofar as data availability permits, this section will cover the following:

i) Analysis of the competitive framework will look at the evolution of the framework of supply in terms of the legal degree of openness, the conditions of competition, the number of players and their behaviour. The purpose of this assessment will be to determine whether there is active competition between suppliers [8]; whether the supply structure is changing; and whether there are barriers to entry such as interconnection problems or discriminatory access. This analysis should disclose the behaviour of suppliers and possible barriers to entry, both at domestic and cross-border level. This latter issue is important, as one aim of the market opening process is to create single markets in network industries. Secondly, the report will assess whether changes in the conditions of competition have been accompanied by changes in the structure of demand. Among other indicators, consumers' behaviour will be closely surveyed.

[8] This analysis does not pre-empt or predicate more detailed analysis of effective competition at the sectoral level, based for example on the existence of a dominant position.

ii) The report will then assess market performance in network industries in a broader sense, as structural changes and increased competition are likely to impact on the various dimensions of market outcomes in these sectors. Besides employment and economic performances, such as prices or productivity, the indicators will look into the fulfilment of general interest service obligations and in particular the quality of services, accessibility conditions, affordability and pricing conditions, environmental performance and impact on social and territorial cohesion. Special attention will be paid to the impact of changes in these sectors on different users and consumers. Given the important territorial dimension in the provision of many of these services, indicators to measure the impact of market performance on territorial cohesion will be developed. Finally, evaluations will assess whether the provisions set in the legislation are meeting their objectives.

The technical annex to this communication includes a preliminary list of indicators based on the information available from sectoral monitoring, plus other indicators that could be developed in the future. This list will be expanded and become more precise as this methodology is gradually developed and implemented.

4.2. Results of the permanent consumer consultation process

This will be a key aspect of the horizontal evaluation process. It will provide valuable information for the development of sectoral policies as well as for the overall assessment of public policies towards network industries providing services of general interest, by assessing whether the evolving performances of network industries are perceived and valued by users.

Two instruments will be developed to monitor the evolution of consumers' opinion about the performance of services of general interest.

i) First, the Commission will regularly produce indicators measuring consumers' satisfaction regarding different aspects of the provision of services of general interest. These indicators will reflect consumers' perceptions based on panel data collected for this purpose. They will take into account price affordability, quality, accessibility, market transparency, registered consumer complaints, contractual relations with providers and regional dimensions of service provision among other issues.

ii) The Commission will also conduct ad-hoc surveys to gather more information from consumers and consumer organisations on specific aspects of services of general interest.

When taking into account consumers' opinions, attention will be paid to different consumer types. In that sense, special attention will be paid to the impact of market opening on those consumers most sensitive to changes in the provision of services of general interest [9].

[9] Special care will be taken in the interpretation of consumer surveys.

The Commission will launch a call for tenders for the external provision of these services in the coming years starting in 2003. Priority will be given to produce data of high quality and extensive coverage.

4.3. Cross-sectoral review of horizontal topics

In this section, the Commission will report on horizontal topics of relevance for all or most of the services covered by the evaluation, that are not the subject of regular sectoral monitoring. It will address issues such as the market opening and reform of network industries and employment, their integration at European level, their environmental impact, productivity and technological change, or their impact on social and territorial cohesion.

These reviews of horizontal topics will provide valuable information with a view to :

i) Providing additional value added to sectoral evaluation and monitoring.

ii) Expanding the range of topics regularly covered in section 1 of the horizontal evaluation reports.

iii) Developing new indicators and data sources for the regular monitoring of services of general interest.

To this end, external studies will be commissioned from experts. In the preparation of this third section of the reports and on an "ad hoc" basis, relevant stakeholders will be consulted.

5. Calendar and transitional provisions

The Commission will produce annual reports implementing this methodology starting in Autumn 2003. It intends to develop and adapt the methodology applied on the basis of the experience gained and taking into consideration comments and contributions from Member States, Community institutions and any other interested parties. In 2006, a more comprehensive assessment will be made to evaluate whether the approach meets the objectives with a view to a possible review.

By the end of 2002, in the context of the Cardiff Process the Commission will produce a working document of the services of the Commission updating the first horizontal evaluation of services of general interest published last year. This update will consist of :

i) A horizontal evaluation of the performance of services of general economic interest based on available indicators.

ii) A new consumers' opinion survey on the provision of those services updating and comparing results with the Eurobarometer published by the Commission in Autumn 2000.

Annex

The qualitative and quantitative data included in the report will be collected from existing data sources using the existing information infrastructure, including the various benchmarking exercises carried out by the Commission's services, and from ad hoc studies. As the lack of data may severely constraint the exercise, close co-operation with data providers -especially Member States, national regulators and stakeholders - is required. In principle, the sectors surveyed in the 15 Member States will include:

- Air transport: segmented as EU versus non EU transport, and as transport of passengers versus transport of freight.

- Electricity: segmented as industrial versus domestic users, and as generation, transmission, distribution and retail supply activities.

- Gas: segmented as industrial versus domestic users, and as production or import, transmission, distribution, and retail supply.

- Postal services: segmented as inland mail (i.e. letters plus direct mails), cross-border mail, inland parcels, and cross-border parcels.

- Railways: segmented as domestic versus cross-border transport, and as transport of passengers versus transport of goods.

- Telecommunications: segmented as local, long-distance and international calls and different types of access, including access to the local loop.

- Local and regional public transport: segmented as bus versus light rail/metro.

The list of indicators below provides a framework for analysis. Some indicators are available, whilst others have to be developed. Furthermore, their relevance and definition will vary across industries. Finally, due to the evolutionary nature of the services provided by network industries, the list of indicator will be subject to constant revision. As a matter of principle, existing data sources and information infrastructure will be given priority for these purposes. The following list is based on desirable indicators, some of which are not yet available.

A. Analysis of the competitive framework.

a) Supply-side aspects of market structure. Quantitative indicators will be used to assess whether competition is developing among suppliers and if competitors are entering the liberalised market.

i) Percentage of demand legally and actually opened to competition.

ii) Date of full legal market opening.

iii) Identification of the incumbent

iv) Number of competitors in the segment.

v) Percentage of the population having the choice between more than x suppliers in the segment.

vi) Market share of the incumbent in the segment.

vii) Concentration ratios.

viii) Internationalisation of companies.

Some other indicators will give information on the structure and regulation of the supply.

i) Degree of vertical integration of the incumbent and other competitors.

ii) Regulator: Identity, powers (right to impose decisions, right to fine, ex-ante or ex-post regulation).

iii) Key facility: identification, type of Third Party Access (regulated versus negotiated and other types of access for specific network segments) for both domestic and foreign competitors, types of unbundling (accounting, management, legal, ownership).

Other indicators should provide information on whether there are any barriers to entry (at domestic level by identifying possible discriminatory access, and at cross-border level). Special attention will be paid to possible interconnection problems between networks due to either physical scarcity or the inefficient use of available capacity.

i) Access charges to the network for domestic and for foreign competitors (including access charges for import, export, and transit).

ii) Structure and transparency of the tarification mechanism.

iii) Time and cost to obtain access to the network.

iv) Percentage of the supply carried out by Third Party Access.

v) Imports as a percentage of domestic consumption.

vi) Interconnection as a percentage of domestic capacity.

vii) Publicity of the available spare capacity.

viii) Percentage of capacity booked by the incumbent (including integrated companies).

b) Demand-side aspects of market structure.

The indicators will analyse whether changes in market structure have impacted on the structure of the demand.

i) Growth of the sector in terms of quantities.

ii) Percentage of consumers having switched suppliers at least once in the market segment open to competition in the last three years.

iii) Price elasticity of demand.

B. Market performance in network industries.

a) Economic and employment performance (paying attention to the regional dimension whenever possible).

i) Contribution of the sector to GDP.

ii) Contribution of the sector to employment.

iii) Evolution of employment in the incumbent.

iv) Evolution of employment in the sector.

v) Productivity of the sector as measured by output per person employed.

vi) Productivity of the incumbent as measured by output per person employed.

vii) Innovation of the sector as measured by patents and R&D effort.

b) Price performance

i) Evolution of prices in domestic currency in real and nominal terms.

ii) Relative price level in euro compared to the EU average.

iii) Affordability: Price of services relative to the income of low/average-income consumers (reported for consumers with different income levels) [10].

[10] When reporting information on consumers and users, differences in consumer types, income levels or regional location will be given insofar as allowed for by data availability.

c) Quality of service.

The quality dimensions of services delivered by industries providing services of general interests vary across industries. Indicators will therefore have to be adapted to the situation in each sector. The indicators below are given here for illustrative purposes. The range of indicators will be expanded subject to data availability.

i) Reliability of services

(1) min/hours/days/weeks of discontinuation or delays in services.

(2) Share of population affected by discontinuation or delays of services within a year

(3) Repair time after service cuts or average length of repair delays.

ii) Accessibility

(1) Time to get connected to the network.

(2) Number of post offices /number of letter boxes per inhabitant.

(3) Number of passengers seats (trains, airlines) per inhabitant.

(4) Access to services in different geographic areas.

iii) Safety.

(1) Number of accidents relative to distance and/or passengers.

(2) Sectoral safety performance indicators.

d) Impact on the provisions of Universal Service Obligations.

i) Network density and evolution (e.g. new or abandoned air transport routes).

ii) Percentage of population not having access to the service (reported by sub-national geographic areas).

e) Environmental dimension

When appropriate, indicators analysing the environmental dimension of the market opening of services of general economic interest will be used. For example:

i) Share of renewable energy sources on gross electricity consumption, by source.

ii) Share of renewable energy sources to electricity generation, by source.

iii) Indices of gross inland energy consumption, by fuel including gas.

Moreover, suitable indicators will be developed in the future to measure the environmental impact of market opening, in particular as regards structural changes in the transport sector having an impact on the environment. These could include the impact of market opening in air transport on emission of greenhouse gases and conventional air pollutants and the impact of structural reforms in various transport sectors on the energy efficiency of the different modes of transport.

C. Consumers' perception of services offered

a) Overall satisfaction with services.

b) Satisfaction with price and quality.

c) Contractual or information problems between consumers and service providers.

d) Indicators of conflict resolution.

e) Reported difficulties switching service providers.

LEGISLATIVE FINANCIAL STATEMENT

Policy area(s): Internal Market

Activit(y/ies): Evaluation of Internal market policies

Title of action: Methodology for the horizontal evaluation of services of general interest

1. BUDGET LINE(S) + HEADING(S)

B5-3001 Implementation and development of the Internal Market

A-7002 Technical and administrative assistance in support of different activities

2. OVERALL FIGURES

2.1. Total allocation for action (Part B): 3.450 EUR million for commitment

2.2. Period of application: 2003-2007

2.3. Overall multiannual estimate of expenditure:

(a) Schedule of commitment appropriations/payment appropriations (financial intervention) (see point 6.1.1)

EUR million (to three decimal places)

>TABLE POSITION>

(b) Technical and administrative assistance and support expenditure(see point 6.1.2)

>TABLE POSITION>

>TABLE POSITION>

(c) Overall financial impact of human resources and other administrative expenditure (see points 7.2 and 7.3)

>TABLE POSITION>

>TABLE POSITION>

2.4. Compatibility with financial programming and financial perspective

[X] Proposal is compatible with existing financial programming.

Proposal will entail reprogramming of the relevant heading in the financial perspective.

Proposal may require application of the provisions of the Interinstitutional Agreement.

2.5. Financial impact on revenue:

[X] Proposal has no financial implications (involves technical aspects regarding implementation of a measure)

OR

Proposal has financial impact - the effect on revenue is as follows:

3. BUDGET CHARACTERISTICS

>TABLE POSITION>

4. LEGAL BASIS

Pursuant to the Interinstitutional Agreement of 6 May 1999 on budgetary discipline and improvement of the budgetary procedure, appropriations entered in this item are intended to finance measures undertaken by the Commission by virtue of its institutional prerogatives.

5. DESCRIPTION AND GROUNDS

5.1. Need for Community intervention

5.1.1. Objectives pursued

The objective is to produce annual evaluations of the performance of services of general economic interest. The European Parliament and the Internal Market, Consumers Affairs and Tourism Council (1.03.2002) have requested these evaluations. The methodology proposed will increase the number of indicators available to measure the impact of structural changes in network industries providing services of general economic interest. It will also produce regular measurements of citizens' sentiment on the performance of these sectors and the impact of market opening.

5.1.2. Measures taken in connection with ex ante evaluation

A summary ex-ante evaluation has been conducted by DG MARKT (available upon request). For that purpose, information provided by other services and technical support from DG ECFIN have been used.

A first horizontal evaluation of serves of general interest was conducted last year in the context of the Cardiff process (SEC (2001) 1998). This first evaluation revealed the availability of quantitative information and helped identify the issues that should be tackled in future and periodic evaluations.

The ex-ante evaluation addressed the following main issues:

- How to expand progressively the availability of indicators, especially as regards qualitative aspects of performance in these sectors-

- What is the best way to produce regular and periodic measurements of consumers' opinion on market performance-

- How to conduct pluralistic evaluations in order to take into account the points of view of stakeholders-

The methodology responds to these questions following a number of criteria stated in the draft communication.

5.2. Action envisaged and budget intervention arrangements

Annual evaluation reports will be produced in the context of the Cardiff process. These evaluations will be submitted to the Internal Market Council, the ECOFIN Council, the European Parliament and the Economic and Social Committee

The evaluations will consist of three parts responding to the three actions foreseen in the evaluation:

1 An analysis of the performance of network industries providing services of general interest based on indicators

2 A survey of consumers' views on the performance of those sectors and the impact of market opening.

3 An analysis of a key horizontal topic that will enlarge the availability of indicators in the future.

All Member States will be covered.

5.3. Methods of implementation

The analysis of performance will be carried out by Commission services based on the information provided by the technical assistance contract.

A database of indicators will be compiled and updated using external technical assistance.

The Commission will produce questionnaires and terms of reference to monitor consumers' opinions and carry out in-depth studies of horizontal questions related to services of general economic interest. For that purpose, two framework contracts will be launched to carry out these tasks on a regular basis.

6. FINANCIAL IMPACT

6.1. Total financial impact on Part B - (over the entire programming period)

6.1.1. Financial intervention

Commitments (in EUR million to three decimal places)

>TABLE POSITION>

6.2. Calculation of costs by measure envisaged in Part B (over the entire programming period)

Commitments (in EUR million to three decimal places)

>TABLE POSITION>

7. IMPACT ON STAFF AND ADMINISTRATIVE EXPENDITURE

7.1. Impact on human resources

>TABLE POSITION>

7.2. Overall financial impact of human resources

>TABLE POSITION>

The amounts are total expenditure for twelve months.

7.3. Other administrative expenditure deriving from the action

>TABLE POSITION>

The amounts are total expenditure for twelve months.

I. Annual total (7.2 + 7.3)

II. Duration of action

III. Total cost of action (I x II) // 0.324 EUR

5 years

1.620 EUR

8. FOLLOW-UP AND EVALUATION

8.1. Follow-up arrangements

Non applicable

8.2. Arrangements and schedule for the planned evaluation

The proposed methodology will be reviewed and evaluated in 2006.

9. ANTI-FRAUD MEASURES

Open tendering procedures will be used for the aforementioned framework contracts.