REPORT FROM THE COMMISSION TO THE COUNCIL, THE EUROPEAN PARLIAMENT, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS - Report on the implementation of the «Culture 2000» Programme in the years 2000 and 2001

TABLE OF CONTENTS

1. Introduction

2. Background of the external evaluation

2.1 the culture 2000 programme

2.1.1. Overview

2.1.2. Selection procedure

2.1.3. Project management

2.1.4. The cultural contact points

2.1.5. Programme dissemination

2.1.6. Complementarity with other community programmes

3. The external evaluation

3.1. The terms of the evaluation

3.2. Methodology

3.3. Results of the external evaluation

3.3.1. Overview of the culture 2000 programme in 2000 and 2001

3.3.2. The financing

3.3.3. Targetting objectives

3.3.4. The beneficiaries

3.3.5. The partnerships

3.3.6. Dissemination of information about the programm

3.4. Quality

3.4.1. Relevance

3.4.2. Output

3.4.3. Outreach

3.4.4. Competence

3.5. Impact

3.6. Efficiency and effectiveness

3.6.1. The efficiency and effectiveness of the projects

3.6.2. The efficiency and effectiveness of the cultural contact points

3.6.3. Efficiency and effectiveness of the programme

4. Main recommendations of the external evaluation and reactions from the commission

4.1. Improving dissemination and information about the programme

4.1.1. Clear dissemination strategy

4.1.2. Probability of obtaining funding

4.2. Improving management of the programme

4.2.1. Closer dialogue with beneficiaries

4.2.2. Improving systematic monitoring of the programme

4.2.3. Upgrading the screening of applications

4.2.4. Paying more attention to the cash-flow problems of the operators

4.3. Improving the selection procedure

4.3.1. Reconsidering the selection procedure

4.3.2. Making visible the various stages of the selection procedure

4.3.3. Improving feedback to refused applicants

4.4. Reconsidering the objectives of the programme

4.5. Improving evaluation of the programme

5. Commission conclusions

EXECUTIVE SUMMARY

This report presents a summary of the main findings and recommendations of the mid-term evaluation of the Culture 2000 Programme, which was carried out by the Danish consultancy PLS Ramboll Management, as well as the main reactions and conclusions of the Commission to these recommendations.

1. EVALUATOR'S FINDINGS

The programming

Culture 2000 followed a comprehensive and coherent logic, aimed at fostering cultural co-operation in Europe. The Programme was complementary to other Community actions and the cultural policies of the Member States. The requirement introduced in 2001 of 5% financial participation by all co-organisers was a good instrument to ascertain the active participation of all cultural operators in the projects, although it may have dissuaded the creation of partnerships with no prior history of co-operation and impede some cultural operators in the associated countries from participating in the Programme. The Commission carried out a number of important activities to disseminate knowledge about the Programme.

The projects

In 2000 and 2001, more than 1 600 applications were submitted to the Programme, of which approximately one quarter received funding. The main beneficiaries were operators from those countries, which also presented the majority of applications. Most selected projects targeted more than one of the specific objectives of the Programme. The number of applications to the Programme declined significantly in 2001. At the same time the number of projects rejected as failing to meet eligibility criteria rose considerably.

The partnerships

Most of the cultural operators receiving funding were relatively small in terms of organisational capacity (budget and staff). The funded projects were managed by a wide variety of organisations (NGOs, national cultural institutions, private enterprises, etc.). The partnerships were mainly formed on the basis of previous co-operation, normally with similar organisations in other countries.

The results

The Programme created cultural added value by creating new forms of cultural expression, attracting greater audiences than planned, and encouraging the movement of artists and cultural operators. Operators completed their projects with the level of quality required. The Programme also succeeded in creating European added value in terms of creating new transnational co-operation and new partnerships that appear to be sustainable. The socio-economic impact seemed to be more restricted.

Efficiency and effectiveness

Efficiency and effectiveness were reached at project level, at the Cultural Contact Points level, at the Programme management level and, therefore, at the overall Programme level.

Recommendations

As for the recommendations, the following table presents a synoptic view of the recommendations of the evaluator and the reactions from the Commission.

Recommendations from Ramboll

// Comments from the Commission

1. Improving dissemination and information about the Programme

a) Clear dissemination strategy (example, targeting the dissemination of information to "core beneficiaries").

b) Informing operators of their relatively high probability of obtaining funding. // 1. Improving dissemination and information about the Programme

a) The Commission already has a clear dissemination strategy (Cultural Contact Points, e-Newsletter, website, etc.), and does not consider it appropriate to target specific beneficiaries, since all operators have the same rights and the Commission has a duty not to treat them in a discriminatory manner.

b) The Commission already gives information about the number of applications as well as the number of pre-selected and selected projects in its website and in its monthly e-newsletter.

2. Improving management of the Programme:

a) Closer dialogue with the beneficiaries, in order to adjust the Programme to their needs. CCPs could be used in this context.

b) Improving systematic monitoring of the Programme, through the creation of a complete and aggregated database.

c) Upgrading the screening of the applications, in order to reduce the number of project applications refused (formal criteria). CCPs could prove useful in this context.

d) Paying more attention to the cash-flow problems of the operators, by reconsidering the 50% before/50% after paying procedure applied to Action 1 projects.

// 2. Improving management of the Programme:

a) A public consultation was launched in the frame of the preparation of the Programme that will succeed Culture 2000. The Forum 2001 also shed light on the needs of the operators. CCPs should not be overloaded with more work than contractually required, otherwise they will not be able to target their activities properly.

b) The Commission will launch in 2004 the new "SYMMETRY" management system, which will address this and other problems.

c) It belongs to the contractual functions of the CCPs to provide technical assistance to applicants. The Commission will inform them of the need to increase further this activity. The Commission also wonders whether the ability to read a call for proposals and fill the applications correctly is not a sign of competence and credibility of the operator that should not be neglected.

d) The cash-flow problems of the operators will be settled, as from 2004 on the payment ratio will be 70/30.

3. Improving the selection procedure

a) Reconsider the selection procedure, in order to make it shorter than 6 months (through a reduction of the delays given to the European Parliament and the Management Committee).

b) Making the various stages of the selection procedure more transparent, by publicising the procedural stages more clearly in the call for proposals, or by supplying information to operators concerning the stage of progress of individual projects.

c) Improving feedback to non-selected applicants: the evaluator recommends that the Commission make the project's evaluation rating of the experts available to the applicant".

4. Reconsidering the objectives of the Programme

This could be achieved by suppressing those related to socio-economic development and/or by prioritising target groups. // 3. Improving the selection procedure

a) These comments will be duly transmitted to the competent institutions.

b) In 2002, the Commission launched a free e-Newsletter, which provides precise information on the progress of applications. The SYMMETRY application will also be useful in this context.

c) The publication of the evaluation rates is possible, and the Commission is working in this sense.

4. Reconsidering the objectives of the Programme

The Programme's current objectives are too broad, especially taking into account the funds available. The choice of objectives is a political one, and the Commission will address this issue at the appropriate moment, namely in the context of the preparation of the cultural Programme that will eventually succeed Culture 2000.

In the light of the evaluation and its conclusions and recommendations, the Commission does not consider it necessary to present a proposal for a modification of the Programme for the period remaining to be implemented.

Moreover, partly for reasons of overall financial planning and partly in order to have the time to present and have considered, through due process, a proposal for a new Culture programme, the Commission has put forward a proposal extending Culture 2000 by two years, to 31/12/2006 [1]. The Commission will present in good time the proposal for the new programme.

[1] COM(2003) [...], [...], p. [...].

1. INTRODUCTION

This report is being presented under Article 8 of Decision 508/2000/EC of 14 February 2000 establishing the Culture 2000 Programme (hereinafter the "Decision") [2], which establishes that no later than 31 December 2002 the Commission shall present to the European Parliament, the Council, the Economic and Social Committee and the Committee of the Regions a detailed assessment report on the results of the Culture 2000 Programme, having regard to its objectives, and accompanied if necessary by a proposal for the amendment of the Decision.

[2] OJ L 63 of 10.3.2000, p. 1.

This assessment report shall, according to the same article, emphasise in particular the creation of added value, particularly of a cultural nature, and the socio-economic consequences of the Community's financial support.

The present report is based on the results of the Interim Evaluation of the Culture 2000 Programme, which was carried out by PLS Ramboll Management in the course of 2002 and 2003.

2. Background of the external evaluation

2.1 The Culture 2000 Programme

2.1.1. Overview

Since 1992, the EC Treaty provides in Article 151 (ex-article 128) for the adoption by the Council, under the co-decision procedure with the European Parliament and after consulting the Committee of the Regions, of incentive measures ("Programmes") to contribute to the European Community's mandate in the area of culture.

These new competencies were executed through the implementation, between 1996 and 1999, of three cultural Programmes: Kaleidoscope [3] (1996-1999), which supported artistic and cultural activities having a European dimension; Ariane [4] (1997-1999), which provided support to the field of books and reading, including translation; and Raphaël [5] (1997-1999), the aim of which was to complement Member States' policies in the area of cultural heritage of European significance.

[3] Decision n° 719/96/CE of the European Parliament and the Council of 29 March 1996 establishing a Programme to support artistic and cultural activities having a European dimension (Kaleidoscope), OJ L 99 of 20.04.1996, p. 20-26.

[4] Decision n° 2085/97/CE of the European Parliament and the Council of 6 October 1997 establishing a Programme of support, including translation, in the field of books and reading (Ariane), OJ L 291 of 24.10.1997, p. 24-34.

[5] Decision n° 2228/97/EC of the European Parliament and the Council of 13 October 1997 establishing a Community action Programme in the field of cultural heritage (Raphael), OJ L 305 of 8.1.1997, p. 31-41.

These "first generation" Programmes provided a useful contribution to the development of Community action in the field of Culture. Taking into account the positive aspects as well as the shortcomings revealed by these Programmes, the Commission decided to replace them in 2000 by a single integrated instrument, the Culture 2000 Programme.

Culture 2000 is a Community Programme established for five years (2000-2004), with a total budget of 167 million euros. It was preceded by pilot actions in 1999 and provides grants for cultural co-operation projects in all artistic and cultural domains (the performing arts, visual or spatial arts, literature, heritage, cultural history, etc.).

According to the Decision, the Culture 2000 Programme aims to contributing to the promotion of a cultural area common to the European peoples. In this context, it supports co-operation between creative artists, cultural operators, private and public promoters, the activities of cultural networks and other partners, as well as of the cultural institutions of the Member States and of the other participant countries, in order to reach the following objectives:

- promotion of cultural dialogue and of mutual knowledge of the culture and history of the European peoples;

- promotion of creativity and the transnational dissemination of culture and the movement of artists, creators and other cultural operators and professionals and their works, with a strong emphasis on young and socially disadvantaged people and on cultural diversity;

- the highlighting of cultural diversity and the development of new forms of cultural expression;

- sharing and highlighting, at the European level, the common cultural heritage of European significance; disseminating know-how and promoting good practices concerning its conservation and safeguarding;

- taking into account the role of culture in socio-economic development;

- the fostering of intercultural dialogue and mutual exchange between European and non-European cultures;

- explicit recognition of culture as an economic factor and as a factor in social integration and citizenship;

- improved access to and participation in culture in the European Union for as many citizens as possible.

According to the Decision, these objectives shall be achieved by the provision by Culture 2000 of financial support to the following actions, which encompass all cultural sectors:

- Actions 1: specific innovative and/or experimental actions, with a duration of one year, and which involve at least three operators from three participant countries. The indicative share of annual funding for these actions shall not be more than 45 % of the total annual budgetary funds allocated to the Programme.

- Actions 2: integrated actions covered by structured multi-annual co-operation agreements, with an approximate duration of 3 years, and which involve at least five operators from five participant countries. The indicative share of annual funding for these actions shall not be less than 35 % of the total annual budgetary funds allocated to the Programme.

- Actions 3: special cultural events with a European and/or international dimension, of non-specified duration and which have no partnership requirements. The indicative share of annual funding for these actions shall be approximately 10 % of the total annual budgetary funds allocated to the Programme. Examples of these actions are the "European Capitals of Culture" or the "European Heritage Days".

In addition to the Actions, Cultural Contact Points were established on a voluntary basis in almost all participant countries. Contracting these Cultural Contact Points, as well as dissemination and information activities account for 10 % of the budget according to the Decision (see point 2.1.4).

The Actions follow either a vertical approach (addressing the needs of one cultural field) or a horizontal approach (addressing the needs of several cultural fields). The indicative allocation of funding to the horizontal approach corresponds to approximately 10 % of the Programmes' financial framework.

Some modifications have been introduced to the Programme between 2000 and 2001, through its calls for proposals published annually.

In 2000, only cultural operators from the Member States and the EEA countries were eligible for funding. In 2001, the 10 associated countries (Bulgaria, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, the Slovak Republic and Slovenia) were included in the Programme.

In 2001, more specific exclusion criteria (based on formal eligibility criteria) were introduced and, in the same year, the so-called "5% rule" was introduced, requiring a minimum of financial participation by project leaders and co-organisers. This rule was introduced to ensure the technical and financial commitment of the co-organisers of the project form the moment of its conception, as the experience from previous programmes had cast doubts on the solidity and sincerity of cooperations beyond the formal application level. The 5 % rule concerns own money or raised and secured money and it is the guarantee of a concrete cooperation.

2.1.2. Selection procedure

The beneficiaries of the Programme are cultural operators. They are selected according to a procedure, which consists of the following steps:

- Fulfilment of technical eligibility criteria: the Commission screens and registers the projects. Those which meet the technical eligibility criteria are marked as "pre-selected". Organisations whose applications are refused are individually informed, and the reasons for the refusal are stated.

- Evaluation of artistic and cultural quality by experts [6], that is, the pre-selected projects are analysed and rated by experts. At their request, experts are given access to projects that were not pre-selected.

[6] The Ministry of Culture in each of the participant countries proposes a list of experts within the relevant artistic fields to assist the Commission in the evaluation of projects. The list of experts is publicised at the end of the selection procedure.

- Proposal by the Commission: the Commission produces a shortlist of the projects proposed for funding based on the evaluation carried out by experts. This list starts from the highest-rated projects downwards, until the budgetary resources are exhausted.

- Acceptance by the Management Committee: the Management Committee, consisting of representatives of the competent national/regional authorities, must approve the shortlist. Only Member States can vote.

- "Droit de Regard" by the European Parliament: the shortlist must subsequently be presented to the European Parliament, which has a month to exercise its "Droit de Regard".

- Verification that the projects to be funded by Culture 2000 have been granted no other funding by the Community, through an inter-service consultation.

- Approval of the project budget finalisations by the financial control services of the Commission.

- Conclusion of contractual agreements with operators. Operators whose projects have not been selected are informed.

- Publication of the list of selected projects.

2.1.3. Project Management

The project managers of the Commission are responsible for the entire process, from the moment of the selection of the project, to the approval of the final report. Each manager is responsible for between 40 and 90 projects. The project managers' contact with the projects during their implementation is normally reactive, although project managers do respond to invitations from project leaders and attend events organised by them.

2.1.4. The Cultural Contact Points

- Cultural Contact Points were established, on a voluntary basis, in almost every country participating in the Programme [7]. The Decision states that the CCPs are responsible for the following activities:

[7] Liechtenstein has not established a CCP.

- Promoting the Culture 2000 Programme;

- Facilitating access to the Programme for, and encouraging participation in its activities by as many professionals and operators in the cultural field as possible, by means of an effective dissemination of information;

- Providing an efficient link with the various institutions providing aid to the cultural sector in the Member States, thus contributing to the complementarity of measures taken under the Culture 2000 Programme and national support measures;

- Providing information and contact at the appropriate level between operators participating in the Culture 2000 Programme and those participating in other Community Programmes open to cultural operators.

The Commission signs a "EU Grant Agreement" with the CCPs defining the roles and responsibilities of these and stipulating a financial compensation for their activities. CCPs convene at biannual formal meetings hosted by the CCP of the country holding the EU Presidency. These meetings are attended by the Commission. Moreover, once or twice a year, CCPs are also invited to Brussels for training and exchange of views.

2.1.5. Programme dissemination

Information about the Programme is available from a different number of sources such as the Official Journal of the European Communities, the Culture 2000 website and the activities of the Cultural Contact Points. Commission's administrators also present the Programme at several occasions and there is also a "Green number" from the Commission to answer questions from the general public. Furthermore, since 2002 more dissemination activities have been added to the above-mentioned. A free e-Newsletter is available to all interested since 2002, and in March 2002 an Internet culture portal was launched presenting all activities of the European Institutions that have a link to culture. The Culture 2000 Programme website informs about funding opportunities and its calls for proposals, and publishes the lists of selected projects.

At the project level, it is the obligation of the project-leaders and co-organisers to disseminate the results of the projects and best practices through available means, such as reports, seminars, CD-ROMs or websites. It is a requirement that the Culture 2000 logo appear on all the output of the funded projects.

2.1.6. Complementarity with other Community Programmes

The European Commission also supports the creative industries and the cultural sector through a number of different Programmes, funds and initiatives [8]. However, none of these Community interventions has cultural co-operation as a general objective, the focus on cultural co-operation being unique to Culture 2000.

[8] For example, the Programmes MEDIA, SOCRATES, LEONARDO DA VINCI, YOUTH and the Structural Funds.

3. THE EXTERNAL EVALUATION

3.1. The terms of the evaluation

In response to the above-mentioned article 8 of the Decision, the Commission (DG Education and Culture, Unit C-2) has launched a call for tenders for the mid-term evaluation of the Culture 2000 Programme (DG EAC/90/01). The contractor selected for the evaluation was the Danish consultancy PLS Ramboll Management. The evaluation was carried between October 2002 and February 2003. The Final Report was sent to the European Commission in June 2003.

According to the Terms of Reference, the scope of the evaluation included: (i) a description of the achievements attained with the support of the Programme, its use of resources, and its results and impact; (ii) an estimate of the impact of the Programme; (iii) an estimate of the efficiency and effectiveness of the Programme, and its interpretation. This should be followed by conclusions and recommendations from the evaluator.

3.2. Methodology

The methodology used by the evaluator included the realisation of interviews with experts, Cultural Contact Points and Commission staff, a survey of project leaders and co-organisers of the projects selected, as well as among refused applicants. On-site case studies were also carried out.

The evaluator themselves recognised that the methodology chosen, like any other methodology, presents weaknesses: "Arguably, the backbone of the evaluation is the survey among the project leaders. This approach was chosen since the projects transform 90 % of the Programme's funds into cultural activities. Therefore, the accuracy and representativity of this data is also of particular importance".

For this reason, and taking into account that the evaluator are more experienced in the sector evaluation than in the cultural sector as such, the conclusions and recommendations presented by the evaluator will be the object of a prudent and careful analysis by the Commission.

3.3. Results of the external evaluation

The evaluator's findings were as follows.

3.3.1. Overview of the Culture 2000 Programme in 2000 and 2001

The Culture 2000 Programme followed a comprehensive and coherent logic, aimed at fostering cultural co-operation in Europe. In the period 2000-2001 the Programme management and specifications underwent some changes. Most notably, the scope of the Programme was widened to encompass the associated countries. Moreover, a minimum of 5 % of financial involvement of the total project budgets was established for all project leaders and co-organisers in 2001, thus ensuring the real and constructive participation of all parties in the project.

In 2000 and 2001, the Programme saw a total of 1 628 applications, 406 of which received funding (approximately 25 %). Of this total of applications 1 514 were submitted by cultural operators in the Member States [9]. In 2001, there was a considerable decline in the number of applications. The number of applications rejected on the basis of formal eligibility criteria was higher in 2001 than in 2000.

[9] Please note that the participation of associated countries only started in 2001.

According to the Commission, the drop in the number of applications is largely due to the fact that many organisations, whose application had been rejected in 2000 as failing to meet the eligibility criteria, realised that their project did not correspond to the spirit of the Culture 2000 programme and refrained from submitting a new application in 2001. Such a fall is in itself of little importance; what matters is the number of "good" and solid applications, that is applications concerning financially viable projects envisaging a real partnership and cooperation and having a sustantial European added-value.

3.3.2. The financing

The 'top five' countries in terms of financing received were France, Italy, Germany, Belgium and Spain, which corresponded approximately to the number of applications submitted by these countries [10].

[10] This concerns the project leaders. The Commission draws the attention to the fact that the value of this statement is limited, since most of the projects (the exception being the translation projects and some Action 3 projects) had to include a partnership of at least 3 operators from 3 different countries participating in the Programme.

The distribution of funds per sector foreseen in the Decision roughly corresponded to the actual funding allocation of the Programme in 2000 and 2001, with cultural heritage and artistic creation being the fields accounting for the largest share of selected projects and the largest amount of overall funding.

3.3.3. Targetting objectives

Most selected projects targeted more than one of the specific objectives in the Programme. The operators neither perceived the objectives as difficult to convert into projects, nor as constraints to their artistic creativity.

Regarding to the Programme's objectives, the Programme was complementary to other community actions and the cultural policies of the Member States.

3.3.4. The beneficiaries

Projects were awarded to a wide spectrum of organisations. However, NGOs were the most frequent project leaders. The majority of the NGOs were fairly small organisations, with an annual budget of less than 250 000 euros and a limited number of employees.

3.3.5. The partnerships

Typically, project partnerships were forged on the basis of previous collaboration (62 %), operators tended to forge partnerships with similar organisations in other countries, cultural operators from the larger countries were the most active in transnational partnerships and cultural operators from all Member States managed projects, while the participation of operators in the associated countries was more uneven.

The above-mentioned "5 % rule" was a good instrument to ensure the active participation of all cultural operators in the project. However, according to the evaluator, the use of the instrument might be a dissuasive to the establishment of new partnerships between operators with no prior history of collaboration. Furthermore, it might impede some cultural operators in the associated countries from participating in the Programme, as it represents a relatively high financial burden for these operators.

3.3.6. Dissemination of information about the Programm

Concerning the dissemination of knowledge about the Programme, the Commission had carried out a number of activities (see point 2.1.5 above), especially in the years after the evaluation period. However, more could be done in order to provide information about the Programme to cultural operators in general, and to applicants in particular.

3.4. Quality

According to the evaluator, a project had to fulfil the following criteria in order to be considered a high-quality project:

- Relevance: the project addressed a real need among cultural stakeholders and was consistent with the Programme's objectives.

- Output: the output was consistent with the resources used.

- Outreach: the project addressed a pre-defined target group corresponding to the resources used.

- Competence: the project was carried out with the appropriate human resources, organisational set-up, and by skilled project leadership.

3.4.1. Relevance

The Culture 2000 Programme was relevant, since surveys carried out by the consultancy showed that the large majority of project leaders and co-organisers felt that their output could not have been achieved without the transnational aspect inherent in Culture 2000 and they would most probably not have realised relevant projects without EC funding.

3.4.2. Output

On aggregate, the vast majority of cultural operators felt that the quality of actual output was in line with their expectations.

3.4.3. Outreach

In examining the outreach of projects, emphasis was placed on the relationship between the planned outreach and the actual output comprising the interface between the projects and their beneficiaries/consumes/users. The evidence revealed that 89 % of the projects produced their planned output.

3.4.4. Competence

In all cases, both project leaders and co-organisers emphasised the high level of competence of their collaborators regarding their artistic/professional skills, which were regarded as enriching for the project's results.

PLS Ramboll's analysis showed that more co-organisers were satisfied with their project leaders' managerial skills (82 %) than with their professional skills (74%). For the project leaders the opposite relationship was true, as they had high regard high regard for the co-organisers' professional skills (90 %) but less so for their managerial skills (67 %).

3.5. Impact

According to the Terms of Reference and the project application forms, three kinds of impact could be inferred to be conceptual elements common to all projects. These were:

- The European added value of the projects: supporting the emergence of European networks, organisers/institutional co-operation and mobility of people and works of art.

- The cultural added value of the projects: promoting new forms of cultural expression and enhancing training and access to culture.

- The socio-economic impact of the projects: fostering initiatives between cultural and social operators to create social inclusion and cultural empowerment for disadvantaged groups and youth.

- The Culture 2000 Programme created cultural added value by creating new forms of cultural expression, attracting greater audiences than planned to its participants' performances and encouraging the movement of artists and cultural operators. The evaluator also concluded that the Culture 2000 Programme succeeded in creating European added value in terms of creating new transnational co-operation and new partnerships that appear to be sustainable. The socio-economic impact of the Programme seemed to have been more restricted, "as approximately half of the project leaders stated that they had not improved cultural access" [11]

[11] The Commission would however like to stress that , according to the evaluator, the "cultural added value" (and "access to culture" is part of it) was high. Moreover, 94 % of the project leaders considered that their projects had improved access to culture. It was only concerning the question of who benefited from the improved access to culture that the project leaders had some problems in identifying the target beneficiaries. The socio-economic impact of cultural projects is therefore impossible to measure.

3.6. Efficiency and effectiveness

The evaluators considered that a project was efficient where results were obtained at a reasonable cost, and effective where the expected results had been obtained and the objectives achieved.

PLS Ramboll provided a short analysis of the efficiency and effectiveness of the projects, of the Cultural Contact Points and of the Culture 2000 Programme.

3.6.1. The efficiency and effectiveness of the projects

The evaluator found that projects were implemented efficiently and effectively. The project leaders reported efficiency in carrying out the projects at a reasonable cost. Furthermore, the project leaders and co-organisers reported that effectiveness was achieved, as the great majority of the projects' objectives had been achieved.

3.6.2. The efficiency and effectiveness of the Cultural Contact Points

Interview and documentary data showed that the CCPs in general had implemented activities agreed in the contracts with the Commission efficiently. This was backed up by data showing that many CCPs pay attention to the issue of cost reduction (electronic distribution of information, application forms, etc.).

Since August 2001, the CCPs also provided assistance for applicants in identifying and locating partners for transnational projects through the Partner Search Mechanism. However, the results from the surveys suggested that partnerships facilitated by the CCPs had been rare.

Moreover, it is the opinion of the evaluator that a more objective but rather rough indication of the efficiency of the CCPs might consist of the fact that, whereas the CCPs were only contractually required to promote the Programme generally and provide assistance to potential applicants, the vast majority of the CCPs had in addition provided assistance to the projects during implementation.

Thus, the analysis of PLS Ramboll Management pointed to the fact that, on the one hand, evidence might suggest that CCPs had more resources available than necessary, or, on the other hand, that the cost-reducing procedures of the CCPs might have allowed them to take on these additional tasks or that they could perform these extra tasks through additional financing from government or private sources.

3.6.3. Efficiency and effectiveness of the Programme

The evaluator considers that the management of the Programme had been carried out efficiently and the Programme had been implemented effectively in achieving its objectives, in spite of the constraints induced by the level of funding. However, some improvements could still be introduced in order to optimise the management and the implementation of the Programme.

3.6.3.1. The selection procedure

It is the analysis of PLS Ramboll Management that the selection procedure in 2000 and 2001 was not as transparent and fast as it could have been, despite the fact that the Commission took all the possible steps in order to shorten the selection procedure.

The evaluator's interviews with cultural operators indicated that the issue was not related to the use of anonymous external experts, but rather to the duration and transparency of the procedure (that is, its "tracking" and "feedback").

The selection procedure was defined in the programming network. In 2000 and 2001 it had taken no less than six months. According to the evaluator, the involvement in the procedure of the European Parliament and the Management Committee created a layer of political involvement in the selection process and slowed down the selection procedure significantly, by approximately two months. While this involvement is a political reality, it lengthened the selection procedure and created undue haste for the operators' preparations.

The matter of the duration of the selection procedure might not only be related to the selection procedure itself, but also to the human resources available. The evaluator noted that the Commission might need additional human resources to handle the work load, if clearly defined deadlines were to be met in the future.

However, the Commission points to the fact that the introduction of the new Financial Regulation and of associated new legal obligations in 2003 will increase the control mechanisms and demands.

3.6.3.2. Monitoring system

The efficiency of Programme management by the Commission could be improved by unifying the existing information concerning projects and operators into one single electronic database. This database would enable the Commission to easily aggregate, extract and analyse statistical information regarding the characteristics of applicants, selected projects and other information vital to monitoring the progress of the Programme.

3.6.3.3. Cash-flow in projects

The financial requirements in general were not overly bureaucratic, in view of the financial audit requirements with which the Commission had to abide. However, the limited organisational capacity of the cultural operators was likely to limit the cash flow of the organisation. The Commissions "50 % before/50 % after" instalment procedure for Action 1 projects (which constitute the majority of the projects supported) presented a cash-flow challenge for many operators. It has to be noted that this "50/50" rule has been replaced in 2003 by a "70 % before/30 % after" rule.

3.6.3.4. Dialogue with operators of the cultural sector

The Commission had demonstrated interest in obtaining advice on how to improve the programming, selection procedures and co-operation with projects in order to improve the Culture 2000 programming. Further consultation with the CCPs could be of benefit the Commission, in terms of acquiring systematic bottom-up feedback from the operators.

3.6.3.5. Prioritising objectives

The objectives of the Programme as stated in the Decision were broad and diverse, with the overarching objective of fostering cultural co-operation in Europe. This had to be put in relation with the funding available to the Programme, which was of 167 million in five years.

There were advantages and disadvantages in having narrower or broader objectives. While narrow objectives might be more targeted and easier to attain, they might be also more constraining for creativity and innovation. Broad objectives had the advantage that cultural operators generally found it easier to convert them into project ideas and did not feel constrained by them, thus enabling new forms of cultural expression and artistic innovation. However, from a policy perspective the sheer diversity of projects, which encompassed several broad objectives led to poor targeting of the objectives, and might have constituted an obstacle to achieving the intended impact

The future challenge lay in creating a balanced Programme, which took into account the special character of the cultural sector and its productions, while adhering to some specific objectives that corresponded with the Community's overall policy framework, not encompassing overly broad objectives.

4. MAIN RECOMMENDATIONS OF THE EXTERNAL EVALUATION AND REACTIONS FROM THE COMMISSION

4.1. Improving dissemination and information about the Programme

4.1.1. Clear dissemination strategy

According to the evaluator, this could be achieved through a clearer dissemination strategy. This should pass through the identification of those cultural operators that are potential beneficiaries of funding and targeting the dissemination of information to the "typical beneficiaries" of the Programme, notably in the associated countries.

Although the Commission shares the view that a clear communication and dissemination strategy is essential to allow the participation of as many cultural operators as possible, it considers that such a strategy already exists (see point 2.1.5 above). This strategy, as the evaluator acknowledges, has been improved over the years of implementation of the Programme, so that many shortcomings at this level that were felt over the first two years of implementation of the Programme do not exist anymore or will disappear very soon (namely through an ever wider distribution of the e-Newsletter created in 2002 and the ongoing restructuring of the Commission's website in a more complete and user-friendly manner).

Concerning the identification of "typical beneficiaries", the Commission wonders whether this is feasible and, especially, appropriate from a political point of view. It is discriminative and undermines the level playing field. In fact, all cultural operators have the same rights, and the Commission insists on treating them in a non-discriminative and equitative manner.

4.1.2. Probability of obtaining funding

According to the evaluator, another way of generating a higher number of applications would be to inform operators of their relatively high probability of obtaining funding, which seemed to be higher than operators think (one in four for the period 2000-2001). It should, however, be explained that these rates depended on the number of applications passing the eligibility criteria, as well as the quality of the individual applications.

The Commission welcomes the idea of getting more applications as higher numbers of high quality projects would crowd out lesser quality projects. The Commission draws the attention to the fact that it already gives information about the number of applications as well as the number of pre-selected and selected projects in its website and in its monthly e-newsletter.

4.2. Improving management of the Programme

4.2.1. Closer dialogue with beneficiaries

A closer dialogue with the beneficiaries of the Programme could provide a further adjustment of the Programme to their needs. Closer dialogue could be attained by making full use of the Cultural Contact Points as "listening posts" through national workshops, user surveys and expert consultations. This form of information flow from the CCPs could thus provide the Commission with valuable information about the Programme at project level, although it has to be kept in mind that thess functions are already part of their remit.

The Commission shares the evaluator's view that a close dialogue with the beneficiaries is essential to improve the Programme. For this reason, the 2001 Forum for Cultural Co-operation in Europe was held, and a comprehensive exercise of public consultation concerning the eventual creation of a new cultural co-operation Programme carried out. Moreover, the Commission attends the formal meetings with the CCPs, keeps regular informal contacts with them and invites them to training meetings in Brussels, thus ensuring a sufficient flow of information. The Commission has also regular contacts, mostly of an informal nature, with the beneficiaries of the Programmes themselves, and is therefore fully seized of their needs and expectations.

Moreover, the extension of the roles of the CCPs to "listening posts" functions might also overburden them, and thus impede them from assuming their contractually defined roles (see below point 4.2.3).

4.2.2. Improving systematic monitoring of the Programme

The evaluator recommends that the Commission create an integrated electronic monitoring system able of bringing together the information that the Commission holds on each years projects, and of enabling statistical information. The monitoring system should include data such as the description of the project, its project leader and co-organisers as well as information on the total budget of the project, the amount applied for, the amount granted, the funding ratio, the evaluation rating and the status of the project.

The Commission currently keeps contact details for organisers, co-organisers and project information via two separate electronic systems, namely one contact database and one database of project fiches. The letters containing the reasons for refusing applicants are also kept. However, the Commission agrees that a single integrated database would be more convenient for all purposes.

For this reason, a webmaster has recently been put in charge of designing an electronic database for the Programme. Moreover, the computer SYMMETRY programme, which will in principle be launched next year [12], will not only allow a better communication between the Commission and the operators and between the operators themselves, but also the improvement of the complementarity between Community Programmes.

[12] SYstem for the Management and Monitoring of Education, TRaining, Youth, Culture and other DG Education and Culture Programmes. SYMMETRY is a Management Information System and Program Management System to be used by the DG Education and Culture, National Agencies, Technical Assistance Offices, as well as by the Executive Agency in the near future. The System will provide all the necessary functionality to actors involved in all aspects of Programme management activities and not only just project management, which means that it will allow to establish work plans, to manage budgets and to give the possibility to create reports in line with the different user profiles (DG Education and Culture, National Agencies, Executive Agency). In particular, the new System will allow to follow up Programmes at centralised and de-centralised level, it will allow the online-submission of applications, it will also serve as communication basis between cultural operators among Europe with a common interest for actions with a European Added Value.

4.2.3. Upgrading the screening of applications

According to the evaluator, it would be important to upgrade the screening of the applications, in order to reduce further the number of project applications refused on the basis of the formal eligibility criteria. Cultural Contact Points could, in this context, provide an initial screening of the project application and appended documentation. Naturally, the responsibility for the application should still remain with the applicant organisation.

The Commission points out that it is already part of the contractual functions of the CCPs to provide assistance to the operators in the submission of their applications. It will be drawn to the attention of the CCPs that this function should be further reinforced, since the evaluation report suggests that CCPs are making enormous efforts in improving the Programme in several domains which are not necessarily their core competence; which is in principle desirable, but may divert their attention from their principal objectives.

It is thus considered that the CCPs should not assume functions with regard to the applications in addition to those currently foreseen, since this could create serious legal problems concerning the responsibility for the correctness of the applications, even if the operators were informed that the responsibility would remain with them, as the evaluator suggests.

4.2.4. Paying more attention to the cash-flow problems of the operators

The evaluator considers that the current payment system of the Commission ("50% before/50 % after) causes cash-flow problems some operators, especially small organisations developing annual Action 1 projects. Therefore, the evaluator recommends that the Commission reconsider this system.

The Commission has initiated a change in the payment procedure. As from 2004, the payment will be 70 % after the signature of the agreement and 30 % after project completion.

4.3. Improving the selection procedure

4.3.1. Reconsidering the selection procedure

The evaluator considers that the overall duration of the selection procedure needs to be reduced. By providing strict deadlines for the completion of the different stages, and by reducing the duration of the Management Committee's and the European Parliament's involvement to two weeks, the selection procedure could be shortened by approximately one month (see point 2.1.2).

The Commission considers this recommendation pertinent, and transmits it to both the Management Committee and the European Parliament, for consideration in the context of the present report.

4.3.2. Making visible the various stages of the selection procedure

PLS Ramboll Management recommends that the Commission take action to create further transparency in the selection procedure. This might be achieved by publicising the procedural stages more clearly in the call for proposals (including approximate dates for completion) or by supplying information to operators concerning its current stage of progress.

In order to increase the transparency of the Programme and the level of information provided to the operators, the Commission in 2002 launched a free e-Newsletter dedicated to the Culture 2000 Programme, which provides precise information on the progress of applications. The information system will be further improved with the introduction of the database SYMMETRY in 2004.

4.3.3. Improving feedback to refused applicants

PLS Ramboll Management recommends that the Commission improve feedback to refused applicants, by making the project's evaluation rating of the experts available to them. Such a feedback procedure would allow cultural operators to improve the quality of their applications in response to the subsequent call for proposals.

The Commission agrees that the publication of the evaluation results would help operators to know "how far" they were from being selected, allowing them to judge the terms of their eventual applications in the framework of a future call for proposals. The Commission will therefore give due consideration to the feasibility of publicising evaluation results.

4.4. Reconsidering the objectives of the Programme

The evaluator recommends that the Commission reconsider the objectives of the Programme. This recommendation is twofold:

- Omission of the objectives concerning the socio-economic development. Focus could be retained on the objectives that directly adress cultural activities and expression in the framework of transnational co-operation; and/or

- Prioritisation of target groups. The evaluator considers that greater targeting could be attained by, for example, changing prioritised target groups from one year to another. According to the evaluator, the focus on specific target groups would allow for greater synergies with other Community action.

The Commission believes that the definition of the Programme objectives is ultimately a political choice, as the evaluator also recognises. The Commission shares the main conclusion of the evaluator that the Programme's objectives currently are too broad, especially taking into account the funds available to achieve them. That is the reason why the Commission envisages reducing the number of objectives in the programme that will replace Culture 2000 from 2007 on and focussing on three main issues : the mobility of people working in the cultural sector, the circulation of works of art and intercultural dialogue. These three objectives have been widely agreed as having a real European added value by cultural stakeholders (see the EP resolutions of 22 October 2002 and of 4 October 2003 respectively on the importance and dynamism of theatre and the performing arts in an enlarged Europe and on cultural industries, as well as the Council Resolution of 19 December 2002 on European added value and mobility of persons and circulation of works in the cultural sector, the conclusions of the Forum on cultural cooperation of November 2001 and the results of the public consultation on a new programme).

4.5. Improving evaluation of the Programme

According to the evaluator, this should include:

- Defining evaluation criteria for the performance of the Commission and the Cultural Contact Points for the remainder of the Programme.

- Monitoring the new steps taken in improving project management. The evaluator recommends that the Commission monitor developments in the following areas of the Programme management: the impact of the "5% rule" on partner co-financing; the impact of the sectoral prioritisation established in 2002; the inclusion of the new Member States in the Programme; the long-term sustainability of Culture 2000 project partnerships; and the long-term sustainability of Culture 2000 project output.

The Commission welcomes the suggestions of the evaluator and will take them into account both in the context of the eventual extension of the Cultural 2000 Programme, and when preparing a new Programme.

The Commission would like to underline that the "5 %" rule was the object of a mature reflection before its introduction. It corresponded to the need to guarantee that partnerships are indeed based on a serious collaboration, thus contributing to the long-term sustainability of both the partnerships and the Culture 2000 projects' output. The Commission is aware of the fact that there are no perfect solutions, but there is no evidence in the report of the evaluator that the "5 %" rule had a negative impact on the functioning of the Programme or on the creation of partnerships.

5. COMMISSION CONCLUSIONS

The importance of the action developed by the Culture 2000 Programme for the promotion of cultural co-operation in Europe (the only Community initiative with such purpose), as well as its importance for the creation of a cultural area common to the European people has been rightly emphasised by the evaluator.

The Commission shares the overall assessment of the evaluator that the Programme has provided a unique and positive contribution to cultural co-operation in Europe, and that projects were characterised by their quality, efficiency, effectiveness and impact. It therefore concludes that the Programme has successfully contributed to the realisation of the objectives established in article 151 of the Treaty. It follows that it is necessary to continue the Community support for cultural co-operation activities in Europe, and that adequate resources should be made available for this purpose.

The Commission agrees that some aspects can be improved; these were referred to in the last chapter.

As far as the Programme's objectives are concerned, the Commission shares the main conclusion of the evaluator that the current objectives may be overly broad, especially taking into account the funds available. The choice of objectives is a political one, and the Commission will address this issue at the appropriate moment, namely in the context of the preparation of the cultural Programme that will eventually succeed Culture 2000.

More generally, the Commission considers that there should be a clear cultural strategy, on which the Programme should draw, and towards its implementation it should work. A cultural Programme for the EU is indispensable, but it would be important to make it consistent with a clear policy framework. A Programme of EU cultural action of this type, which would complement policies and actions in other domains, increase real partnerships, and enhance the sustainability of created European cultural added value is the strongest argument for adequate resources and for developing new partnership models.

According to these principles, the special cultural events [13], that is, the emblematic actions of special importance which allow the celebration of a common European cultural space (the so-called Actions 3), should be kept and reinforced.

[13] The analysis of these actions as such is very limited in the report from PLS Ramboll Management.

The Commission does not consider it appropriate to present proposals for the modification of the Culture 2000 Programme at the present stage, for the following reasons:

- The vast majority of the recommendations of the evaluator with which the Commission agrees do not require the modification of the Culture 2000 Programme, but only its better use and management;

Moreover, partly for reasons of overall financial planning and partly in order to have the time to present and have considered, through due process, a proposal for a new Culture programme, the Commission has put forward a proposal extending Culture 2000 by two years, to 31/12/2006 [14]. The Commission will present in good time the proposal for the new programme.