52004DC0152

REPORT FROM THE COMMISSION - Interim report on the implementation of the second phase of the Leonardo da Vinci Programme (2000-2006)

1. Introduction

Council Decision 1999/382/EC of 26 April 1999 [1] establishes the second phase 2000-2006 of the Leonardo da Vinci programme for the implementation of a Community vocational training policy. Article 13 of that Decision requires the Commission to submit an interim report on the implementation of the programme by 30 June 2004.

[1] OJ L146, 11.6.1999, p33

The purpose of this communication is

- to provide a better understanding of what has been achieved in the period 2000-2003, how it has been done, and to what extent the initial objectives are being met;

- to identify ways to improve the implementation of the programme in the period 2004-2006;

- to contribute to the development of the Commission's proposal for a new generation of education and training programmes (2007-2013).

This communication is based on three main inputs:

- reports and data available within the Commission services

- an external mid-term evaluation for the period 2000-2002, undertaken by the company Barbier Frinault & Associés [2]

[2] Report and executive summary are available at http://europa.eu.int/comm/education/ programmes/evaluation/evaluation_en.html)

- a series of national reports on the implementation of the programme in the participating countries [3].

[3] Reports received from Austria, Bulgaria, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Liechtenstein, Lithuania, Luxembourg, Norway, Poland, Portugal, Romania, Slovakia, Spain, Sweden, Tchec Republic, The Netherlands, United Kingdom.

2. The 2nd phase of the Leonardo da Vinci programme - overview and context

Leonardo da Vinci is now in its second phase (2000-2006) and builds on the experiences of the first phase (1995-1999).

The programme is open to the EU Member States, to the EEA/EFTA countries (Iceland, Liechtenstein and Norway), to the Accession Countries, to the pre-accession countries and, it is expected that Turkey may participate from 2004.

The overall objective of the programme is defined in Article 1 of the decision:

"This programme shall contribute to the promotion of a Europe of knowledge by developing a European area of co-operation in the field of education and vocational training. It shall support Member States' policies on lifelong learning and the building up of the knowledge and skills and competences likely to foster active citizenship and employability."

The general and specific objectives, specified in Article 2 of the Decision, are:

G1. to promote employability and facilitate vocational integration and reintegration;

G2. to increase and develop adaptability, particularly in order to consolidate technological and organisational change;

G3. to improve competitiveness and entrepreneurship, also in view of new employment possibilities;

and

S1. to improve the skills and competences of people, especially young people in initial vocational training at all levels;

S2. improve the quality of, and access to, continuing vocational training and the lifelong acquisition of skills and competences;

S3. promote and reinforce the contribution of vocational training to the process of innovation.

The external evaluation acknowledges that the Decision of Leonardo da Vinci II is more easily comprehensible than Leonardo da Vinci I, mainly due to a substantial reduction of the number of specific objectives and of the implementation measures. However it underlines that the specific objectives are not quantified (no indicators), difficult to measure, and that the general objectives seem too ambitious in relation to the means available. This makes the evaluation of the programme more difficult.

The Council decision for the programme foresees a budget of 1.150 MEUR for the seven years of Leonardo da Vinci phase 2. Additional contributions from the EFTA, accession and pre-accession countries complement the budget (detailed figures for 2000-2003 are in annex 1).

The programme is managed by the services of the European Commission (some 60 Staff) with the support of a Technical Assistance Office (some 10 Staff) and of National Agencies (32 Agencies with around 390 Staff).

The political context

Subsequent to the adoption of Leonardo da Vinci II in April 1999, a number of policy initiatives have been launched with a major impact on vocational training policy and consequently on Leonardo da Vinci II : the Lisbon strategy of March 2000, which accords an important role to education and training for the achievement of its goals [4]; the communication of the Commission on 'Making a European Area of Lifelong Learning a Reality' [5], setting a new paradigm for education and training; the report of the Education Council and the European Commission on the common objectives of education and training systems [6], and the subsequent detailed work programme [7]; the Council resolution of November 2002 on enhanced European co-operation in Vocational Education and Training (VET) [8] extended by the subsequent Copenhagen declaration that was adopted by 31 Ministers, the Commission and the social partners at European level; the Action Plan for the promotion of language learning and linguistic diversity [9], adopted by the Commission in July 2003 and based on the experience of the European Year of languages.

[4] Lisbon European Council, 23-24 March 2000 : Presidency Conclusions

[5] COM(2001) 678 final

[6] COM(2001) 59 final

[7] 6365/02 EDUC 27

[8] 14343/02 EDUC 144, SOC 521

[9] COM(2003) 449 final

The external evaluation acknowledges that, although all these policy initiatives took place after the adoption of Leonardo da Vinci II, the legal basis of the programme made it possible to cope with these new challenges to a large extent.

The emergence of these major political initiatives in the area of education and training might point to one important impact of the European education and training programmes: Many key actors (national and European) consider that these political initiatives and advances would not have been possible without the existence of and contributions from the European education and training programmes.

3. Implementation of the programme

3.1. Architecture of the programme

The Decision foresees seven measures:

a) Trans-national mobility of persons undergoing vocational training;

b) Pilot projects based on trans-national partnerships;

c) Promotion of language competence and understanding of different cultures;

d) Development of trans-national networks;

e) Development and updating of reference material;

f) Joint actions with other Community programmes;

g) Accompanying measures

These measures are grouped together under three procedures, reflecting different management approaches: decentralised, centralised and mixed.

>TABLE POSITION>

The range of procedures and the fact that some measures are allocated to two different procedures make the management of the programme complex. In addition, procedure B foresees a sharing of responsibilities between European and national levels, which has lead to a number of problems in the implementation of the programme.

3.2. Management structures

The main actors are: the European Commission, the Technical Assistance Office, the National Authorities and the National Agencies [10]. Around 83% of the programme's budget is managed at national level.

[10] Commission Decision of 18 February 2000 laying down the provisions governing the responsibilities of the Member States and the Commission as regards the National Agencies in the context of the general guidelines for implementing the second phase of the Leonardo da Vinci programme [not published in the Official Journal].

The Commission is responsible for ensuring the implementation of the programme in accordance with the Council Decision, supported by the Technical Assistance Office (TAO).

The National Authorities are responsible for the efficient implementation of the programme at national level. They are supported in this task by the National Agencies, which are co-funded by the European Commission and the National Authorities.

The Programme Committee is composed of two representatives from each participating country and chaired by the Commission. Representatives from the Social Partners participate in the Committee as observers.

The external evaluation at European level and the 'national reports on the implementation of the programme' all underline the positive impact of the decentralisation of a substantial part of the programme's activities on the implementation of the programme.

3.3. Calls for proposals

The programme is implemented by means of calls for proposals. The first call was published in 1999 and covered 2000 to 2002. The second call, published in 2002, covered 2003 and 2004. New priorities have been defined, reflecting the new paradigm for education and training as defined in the Commission's communication "Making a European Area of Lifelong Learning a Reality". The last call for 2005-2006 will be published in 2004.

These calls for proposals make it possible to a certain degree to take into account policy developments in the area of vocational education and training, including the latest changes in the field of EU language policy.

The external evaluation, however, stresses that the time period to address new developments can be rather long (validity of calls is respectively 3, 2 and 2 years). In addition it underlines that the calls for proposals should be used to provide additional focus to the programme in terms of target groups and areas of intervention.

It has to be mentioned nonetheless that the yearly publication of calls for proposals was criticised under Leonardo I, and calls valid for 2 to 3 years are regarded as an improvement since they offer more stability.

3.4. The evaluation and selection of proposals

The actors concerned are the Commission services, the National Authorities, the National Agencies and the Leonardo Committee. Their respective roles and responsibilities are defined at a general level in the annex to the Decision. Several complementary documents describe more in detail the rules and procedures to follow in relation to the evaluation and selection of proposals.

Compared to Leonardo da Vinci I, the existing procedures (A and B) have been further decentralised, an evaluation process in two steps has been introduced, and a completely centralised procedure has been added.

3.4.1. Procedure A - mobility

Under Procedure A, the National Authorities and Agencies are in charge of evaluating and selecting the complete set of mobility projects. The European dimension of this procedure is ensured through the use common selection and evaluation procedures and criteria defined by the Commission services in co-operation with the Leonardo Committee.

The external evaluation underlines that the evaluation and selection process under procedure A is appreciated by all actors. The project promoters appreciate in particular the short time period between proposal submission and selection decision. However, the administrative formalities, i.e. the submission and evaluation forms, are considered too complex and detailed.

3.4.2. Procedure B - Pilot projects, trans-national networks, language competence

Under Procedure B, the responsibility for implementing the actions concerned is shared between national and Community levels. Pre-proposals are evaluated and selected at national level exclusively, while full proposals are evaluated at both national and community levels, and the selection decision is taken by the Commission services after consultation with the Leonardo Committee. Unfortunately the Decision text does not define the roles of the different actors very precisely, thereby leaving for substantial space for interpretation.

The external evaluation highlights "lack of clarity in the decisions regarding the exact rules applicable to the evaluation of proposals by the national agencies, which are in competition with each other for the allocation of the overall budget for interventions for Procedure B projects. This results in different practices in different agencies, which has an adverse impact on the clarity and uniformity of this evaluation and selection procedure. These diversified practices involve potential risks of unequal treatment of promoters from different countries, especially when certain national agencies call on their own personnel to evaluate proposals, while these agencies also provide the same promoters with advice at the time of preparation of their proposal."

The external evaluation however acknowledges the fact that the Commission services, in co-operation with the Leonardo Committee, produced in July 2003 a code of good practices to address the issue. In November 2003, the Committee adopted a positive opinion on these improved evaluation and selection procedures.

Finally, project promoters are positive towards the two-stage approach (pre-proposals and full proposals) and towards the increasing importance of quality, as the paramount evaluation and selection criterion. However they also see the two-stage procedure as a reason for the long delay between proposal submission and contract signature, and as a source for a number of cumbersome administrative formalities.

3.4.3. Procedure C - Reference material, projects submitted by European organisations

Under Procedure C, the Commission services have the responsibility for the evaluation and selection process, implemented through a peer review system.

According to the external evaluation, the promoters are satisfied with the evaluation and selection process for Procedure C proposals, they appreciate in particular the two-stage approach and the focus on quality.

However, as for Procedure B, they also see it as a reason for the long delay between proposal submission and contract signature, and as a source of a number of cumbersome administrative formalities.

3.5. Information and communication policy

Information and communication activities relating to the programme and its calls are shared between the Commission services and the National Agencies. The role of the National Agencies is becoming more important through the increased decentralisation of the programme management. The Commission services and the National Agencies use a number of media and distribution channels to reach their target audiences. The web-sites at European and national levels are increasingly important. Publications of brochures and information letters, organisation of and participation at seminars, conferences and information days are the other main tools used.

The external evaluation stresses that the procedures applied in the area of information and communication are appropriate, in particular in light of the high level of satisfaction expressed by the promoters. However it stresses also that it could be improved through "a consistent and co-ordinated strategy for promoting and presenting the programme at the level of all three procedures", which should lead to an increased visibility of the programme and to an improvement in the quality of the projects and activities.

3.6. Advice and assistance to promoters (prior to submission of proposal)

The National Agencies are the main providers of advice and assistance to potential promoters of projects under procedures A and B. This support is highly valued by the promoters (external evaluation), even if there are substantial differences between approaches in the different countries.

However few National Agencies advise potential promoters of Procedure C projects. Indeed, those National Agencies that do not advise their national promoters for procedure C proposals consider that it is the role of the Commission services to provide this advice.

"The allocation of responsibilities between Commission services and the agencies with regard to the provision of assistance to promoters of procedure C projects needs to be clarified. An improvement in the advice provided to promoters of procedure C projects should boost the quality of the proposals submitted and enable more of them to be selected." (external evaluation)

The Commission services plan to organise each year in January, starting in 2004, an information day for all interested promoters of procedure C proposals. The first such information day, which has been organised on 13 January 2004 in Brussels, was highly appreciated by the participants.

3.7. Monitoring and follow-up of projects

This activity is crucial for the programme's success. It covers on the one hand the formal or contractual aspects of the projects including contract signature, submission and acceptance of deliverables, payments, contract amendments if required; and on the other hand monitoring of the content monitoring, notably through the thematic monitoring of projects in the same field, with a view to ensuring the highest possible quality outcomes of the projects, and the best possible impact for the programme.

The National Agencies monitor and follow-up Procedure A and B projects, while the Commission services focus on the Procedure C projects and the overall co-ordination of the thematic monitoring of procedure B and C projects.

The external evaluation stresses that "the procedures applied by the Agencies in the area of individual monitoring of procedure A and B projects, and by the Commission and the TAO in the area of individual monitoring of procedure C projects, are appropriate: according to the promoters themselves, the support which they receive is effective and the evaluation of the progress achieved by promoters (evaluation of interim and final reports and on-site visits) appears to be carried out properly."

Launched in 2002 at the initiative of the European Commission the initiative 'Thematic Monitoring' aims to strengthen the exchange of experience and networking between projects, practitioners and decision-makers. It contributes to raising the visibility of projects' achievements and to the dissemination of their results.

"The procedures applied in the area of collective thematic monitoring are greatly appreciated by all the players concerned. Its usefulness as an ex ante valorisation measure is emphasised. However, the players regret that these procedures are restricted to procedure B and C projects only." (External evaluation)

3.8. The dissemination and exploitation of project results (Valorisation)

Dissemination and exploitation of project results have been a major point of criticism in the ex-post evaluation of the 1st phase of the Leonardo programme.

Therefore the Commission services established a specific unit within DG Education and Culture to work on the dissemination and exploitation of project results. The activities of the first three years (2000-2002) focused on project results from Leonardo I, based mainly on an ex-post approach (support of measures targeting the dissemination and exploitation of existing results from completed projects). A database of good practices has been developed and a number of thematic brochures have been published featuring examples of good practice, stemming mainly from the Leonardo da Vinci I programme.

At the beginning of 2003, the Commission services have developed a new strategy for the pro-active dissemination and exploitation (valorisation) of projects while they are still running. This ex-ante valorisation strategy considers dissemination and exploitation as an integral part of the project life-cycle.

The external evaluation considers this new strategy as a big step forward. However it underlines also that "the valorisation initiatives launched at European level and the tools for the dissemination of products and best practices need to be adapted still further to the needs of target groups and programme users."

The launch of a specific call for proposals on innovation transfer in October 2003 is a first step to address this request (see also point 4.2.4).

Some national reports suggest the creation of a specific fund for successful pilot projects to support exploitation and dissemination related activities.

3.9. Administrative management of the programme

The external evaluation concludes that the administrative management has improved considerably since Leonardo I:

- "Clear administrative procedures have been developed for the management of project funds,

- Clearly specified time periods have been laid down for implementing procedures,

- Management of procedure A and B projects has been decentralised and is now handled at the Agency level,

- The Agencies' activities are supervised via a system of regular feedback and monitoring by Commission personnel,

- The Technical Assistant Office assists the Commission with procedure C proposals and projects but has no decision-making authority."

However the cumbersome and time consuming feedback system and the lack of procedural flexibility are a major concern for all parties. (external evaluation and national reports)

In this context, many promoters and National Agencies feel that the introduction in 2003 of the New Financial Regulation (NFR) has not resulted in a simplification of the project management. Certain rules make the participation of a number of organisations, in particular SMEs, in the programme more difficult, as more steps are added to the verification process before contracting and during the project lifetime.

4. Achievements, results, impact by individual measure

4.1. Mobility projects

The mobility measure provides support for trans-national placement projects for trans-national exchange projects. The individual projects are implemented by organisations (not individuals) that organise the placement or exchange of the respective beneficiary.

4.1.1. Facts and first results

Annex 2 provides a detailed overview on the implementation of this measure during 2000 - 2003.

For 2000-2003, an overall budget of around 300 MEUR has been committed for some 7,000 placement projects and some 2,300 exchange projects. These projects provide for the placement of 143,000 people undergoing initial vocational training, students, young workers or recent graduates and the exchange of 22,000 human resources managers in the business sector, vocational training programme planners and managers, particularly trainers and mentors, and occupational guidance specialists.

As mobility projects normally cover two years, the definitive numbers of beneficiaries of placements and exchanges, and the final budgetary figures are available to the Commission services only after a certain period of time. The final numbers and figures for mobility projects contracted in 2000 show that some 85% of the initial budget was effectively spent, and 95 % of the planned placements/exchanges took place.

13.3 % of the beneficiaries in placement/exchange projects came from SMEs and a minimum of 30 % performed their placement/exchange in an SME. The latter figure is probably nearer to 50 %, as many replies (38.7 %) indicate Vocational Training Institutes as host organisations, and these organisations act in practice as intermediaries for the placement/exchanges of beneficiaries in other organisations, mainly SMEs.

The main language used in the framework of placements and exchanges is English, followed by German and French.

Slightly more women (51.5%) benefit from placements and exchanges than men (48.5%). A more detailed gender breakdown will be provided in the final programme evaluation.

The number of handicapped persons participating in mobility actions is some 2 %.

4.1.2. Assessment

This measure is as highly valued in the external evaluation report as in the national reports. The measure is relevant in relation to the needs of the target groups and to the achievement of the objectives of the programme.

The external evaluation and the national reports conclude also that the mobility measure is highly effective as well in view of the related operational objectives, of the results of the projects and its contribution to the achievement of the specific objectives of the programme.

"beneficiaries and projects record improved core and key skills, increased confidence levels and sense of personal responsibility, development of new language skills and exposure to new culture" (national report - UK)

"Une mesure qui favorise l'insertion professionnelle... A titre d'exemple, le taux de réinsertion des demandeurs d'emploi participant au programme est de 100%" (Contribution de la France sur l'avenir des programmes communautaires)

"Thanks to the development of an intensive, diversified and high-quality 'mobility' activity, under the impetus of the national agencies and Commission services, Leonardo has, in general, "strengthened the competencies and skills of people, especially young people" over the period 2000-2002." (External evaluation)

The complete decentralisation of the mobility measure and, in particular, its administrative management, is seen very positively. The main weak point, particularly underlined in the external evaluation report, is the absence of a common software tool to provide efficient, rapid and reliable feedback.

4.2. Pilot projects

Pilot projects involve trans-national partnerships (at least three organisations from three different countries) in the development and transfer of innovation and quality in vocational training, including actions aiming at the use of information and communication technologies (ICT) in vocational training. These projects are managed in a "semi-decentralised" way.

4.2.1. Facts and first results

The tables in Annex 3 provide a detailed overview on the implementation of this measure during 2000 - 2003.

During the first four years (2000-2003), 825 pilot projects were supported, representing an overall commitment of 271 MEUR. The average total cost of a project is of 470,000 EUR with a financial contribution per project of 330,000 EUR (~ 70%). The maximum EU contribution for projects is set to 600,000EUR (max of 200,000 EUR/year for 3 years). The projects last in average 2,4 years and involve more than 8000 organisations (average/project: 10 organisations from 5 countries).

As specified in the Decision, the programme pays particular attention to people at a disadvantage in the labour market, including disabled people, to practices facilitating their access to training, to the promotion of equality and to equal opportunities for women. According to the external evaluation,

"33% of the projects include specific measures to address the needs of people at a disadvantage

27 % of the projects include specific measures to facilitate the access of handicapped persons to training and work

27 % of the projects include specific measures to facilitate the access of women to training and work."

Through this measure the programme seeks to foster co-operation between vocational training institutions, including universities, and undertakings, in particular SMEs.

Vocational training institutions are heavily involved in pilot projects (in 27% of the projects as promoters, in 23% as partner). Universities are project promoters in 19% of the projects, and participate as partners in 16% of projects. SMEs act in 9% of the projects as project promoter and in 16% as partner organisation. Large companies are involved in pilot projects only to a limited degree (in 1% of the projects as promoter and in 4% of the projects as partner organisation).

4.2.2. Assessment

As with the mobility measure, pilot projects are also highly valued by the external evaluation and the national reports. The measure is very relevant in relation to the needs of the target groups and to the achievement of the objectives of the programme.

The external evaluation and the national reports conclude also that the pilot projects are highly effective in view of the related operational objectives. At the end of 2003 few projects were fully completed, so it was not possible to evaluate the results of the projects and their contribution to the achievement of the specific objectives of the programme. However expectations are high both in the external and in the national reports. In particular, the latter state that the quality of pilot projects has improved, thanks to their close follow-up by the National Agencies.

The external evaluation points also out that the programme should have a clearer approach to "innovation transfer". Proposals are assessed for their innovative character. Whereas product innovation is of interest to all countries, process innovation aimed at modernising a given training system is a 'relative' concept, and what is innovative in one country, is not necessarily so in another. Therefore the report recommends that transfer of innovation between participating countries, both product and process innovation, should be financed by the Leonardo programme on the same basis as a 'absolute' innovation, but subject to specific terms and conditions adapted to the particular need.

The Commission services share this view and have already launched a specific call on innovation transfer in 2003. See also point 3.2.7.

The external report questions also the maximum ceiling of funding for pilot projects that focus on product innovation. Indeed it appears that the development of increased European co-operation in the area of vocational training is leading to a need for large-scale projects, which can be developed rapidly in order not to become obsolete.

The Commission services share this view. However this recommendation can not be taken up in the framework of Leonardo da Vinci II, since it would require a review of the legal basis. This recommendation will however be considered when drawing up the proposal for the new generation of education and training programmes.

4.3. Language Competence

This measure provides support for trans-national projects to develop language competences in a vocational training context. Special attention is given to projects on less widely used and taught languages.

Projects under this measure are implemented in the same way as pilot projects, i.e. proposals submitted by 'European' organisations follow the provisions of Procedure C, whereas the other projects fall under Procedure B (semi-decentralised approach).

4.3.1. Facts and first results

During the first four years (2000-2003), 88 language competence projects were supported, representing an overall commitment of 26.7 MEUR. The average total cost of a project is 430,000 EUR with a financial contribution per project of 304,000 EUR (~ 71%). The maximum Community contribution for projects is 600,000 EUR (200.000 EUR per year for 3 years). The projects last on average 2.3 years and involve around 800 organisations (average/project: 9 organisations from 5 countries). According the external evaluation

"41% of the projects include specific measures to address the needs of people at a disadvantage;

6 % of the projects include specific measures to facilitate the access of handicapped persons to training and work;

18 % of the projects include specific measures to facilitate the access of women to training and work."

As for the pilot projects, vocational training institutions are heavily involved in language competence projects (in 29% of the projects as promoters, in 28% as partner). Universities are project promoters in 23% of the projects, and participate as partners in 23% of projects. SMEs act in 14% of the projects as project promoter and in 17% as partner organisation.

4.3.2. Assessment

'To promote language and cultural competences in vocational training' is highly relevant in the European context and in relation to the needs of the target groups. However, following the external evaluation it is less clear to what extent the measure and its definition and implementation in the context of the Leonardo da Vinci programme is adequate to meet these needs. The value and the goal of the measure do not seem to be understood by the potential project promoters. They consider 'language competence' projects as pilot projects; given that the budget available to this measure is far smaller than the budget allocated to pilot projects, this measure is considered as less important, and proportionally far fewer proposals are submitted. This is reinforced by the attitude of the National Agencies that promote pilot projects much more intensively than 'language competence' projects.

As result, the minimum budgetary target set at 5% in the decision has not yet been reached. From 2000 to 2003, this measure has consumed less than 4% of the overall budget. The external evaluation concludes that this measure has reached only partially its operational objective over the first four years of the programme.

The external evaluation recommends the integration of the measure 'language competence' into 'pilot projects' ("l'intégration des mesures compétences linguistiques et réseaux transnationaux dans la mesure projets pilotes est proposée", external evaluation, p.58). However, it acknowledges at the same time that some further analysis might be required as the proposed solution was already implemented under the first phase of the Leonardo da Vinci programme and did not work any better then.

A separate, external evaluation [11] on the impact of language measures in the Leonardo and Socrates programme underlines the relevance and effectiveness of both programmes, in particular in creating a positive attitude to language learning and in contributing to better intercultural comprehension and greater openness.

[11] Evaluation of the extent to which the Socrates and Leonardo da Vinci programmes have achieved the programmes' linguistic objectives, Deloitte & Touche, Final Report, December 2003

4.4. Trans-national networks

This measure provides support for bringing together relevant players in the area of vocational education and training. These networks contribute to identifying and collecting, distilling and building on European expertise and innovative approaches to the analysis and anticipation of skills requirements and to disseminating the outputs and project results.

4.4.1. Facts and first results

During 2000-2003, 45 trans-national networks were supported, representing an overall commitment of 13.1 MEUR. The average total cost of a trans-national network is of 585,000 EUR with a financial contribution per project of 290,000 EUR (~ 50%). The maximum Community contribution for projects is 450,000 EUR (150.000 EUR per year for 3 years). The networks last in average 2.7 years and involve around 720 organisations (average/project: 16 organisations from 9 countries). According to the external evaluation,

'18% of the projects include specific measures to address the needs of people at a disadvantage

18% of the projects include specific measures to facilitate the access of handicapped persons to training and work.'

22% of the members of the trans-national networks are vocational training institutions, 14% are universities, 11% are SMEs.

The main activity of the networks concerns the exchange of experience and the strengthening of innovation in the field of vocational education and training in bringing together experts or organisations specialised in one specific topic. Another important aspect concerns the dissemination of project results, mainly owned by the members of the network. The main direct targets in this dissemination work are SMEs and employers.

4.4.2. Assessment

Trans-national networks are relevant and address a clear need in the field of vocational education and training in Europe. The new political context and its focus on reinforced European co-operation in the field make this instrument still more relevant.

Unfortunately, in contrast to the pilot projects, this measure does not seem to be well understood by the target groups. This together with the lower rate of co-funding (max. 50%) seems to be the main reason why only a small number of good applications were received. From 2000 to 2003, this measure consumed less than 2% of the overall budget. The external evaluation concludes that this measure only partially reaches its operational objective.

The specific role of networks needs to be better developed and explained. In particular networks' complementarity to 'normal' projects and the possibilities of synergies and mutual enrichment between them have to be strengthened.

4.5. Reference Material

This measure aims at the development and updating of reference material through support for surveys and analyses, the establishment and updating of comparable data, the observation and dissemination of good practices and the comprehensive exchange of information.

Projects under this measure should contribute to

- establishing comparable data on vocational training systems and arrangements, practices and various approaches to qualifications and competences in the Member States, or

- producing quantitative and/or qualitative information, analyses and observing best practice in support of policies and vocational training practices for lifelong learning that cannot be made available by Eurostat or CEDEFOP.

4.5.1. Facts and first results

From 2000 to 2003, 46 trans-national 'reference material' projects were supported, representing an overall commitment of some 19 MEUR. The average total cost of a project is of 520,000 EUR with a financial contribution per project of 410,000 EUR (~ 79%). The maximum Community contribution for projects is 900.000 EUR (300.000 EUR per year for 3 years). The reference material projects last in average 2.4 years and involve a total of 500 organisations. According to the external evaluation,

'33% of the projects concern specific measures in the field of vocational education and training for people at a disadvantage

33% of the projects concern specific measures in the field of vocational education and training for handicapped persons

11% of the projects concern specific measures in the field of vocational education and training for women.'

Universities or Research Centres are the promoting organisation of almost 2/3 of the 'reference material' projects.

4.5.2. Assessment

The 'Reference material' projects address a well identified need in the field of vocational education and training in Europe. Public and private decision-makers need to have access to comparable, quantitative and qualitative information about vocational training in Europe.

A limited number of high quality 'reference material' proposals have been submitted during the first 4 years. From 2000 to 2003, this measure consumed less than 3% of the overall budget. The external evaluation concludes that this measure only partially reaches its operational objective.

The "bottom-up" approach via calls for proposals did not work as expected. The large majority of reference projects submitted were based on qualitative approaches and only one project from 2000-2002 was based on a quantitative approach.

Therefore the Commission, in close co-ordination with the Leonardo da Vinci Committee, decided in 2002 to modify the approach, and to launch a number of calls for tenders to collect the required statistical data. First contracts under this scheme have been signed in 2003.

4.6. Joint Actions

This measure provides support for joint projects with other Community programmes and initiatives promoting a Europe of knowledge, particularly with the Community programmes in the fields of education and youth.

Joint actions are managed at European level; they are steered via common calls for proposals for selected themes of interest in fields, which are not exclusively covered by one single programme. The themes, the annual budget and the proposed list of selected projects in relation to joint actions are submitted to the respective committees for opinion.

4.6.1. Facts and first results

So far three common calls for proposals for selected themes, involving the Youth, Socrates and Leonardo da Vinci programmes, have been published: In total 37 joint projects have been supported, representing an overall budget commitment of 7.6 MEUR. The Leonardo da Vinci programme provided 34 % of the funds (2.6 MEUR). The average total cost of a project is 298,000 EUR with a financial contribution per project of 205,000 EUR (~ 69%). The joint action projects of 2001 last on average 16 months, the projects of 2002 and 2003 last 2 years; in total they involve for the three years some 300 organisations.

4.6.2. Assessment

This measure provides the opportunity for the Leonardo da Vinci programme to launch a number of projects on specific themes in co-operation with other Community programmes. So far the experience has been limited to co-operation projects with the Youth and Socrates programmes. Such projects make it possible to bring together actors from different domains to work together on a topic that is of high relevance for each of them. These co-operations help to promote the approach of lifelong learning in a very concrete way.

However the implementation of this measure is cumbersome, due to the involvement of three programme committees, although a common sub-committee has been established to facilitate the work.

5. Conclusions

The 2003 external evaluation and the national reports reflect in general a very positive view of the 2nd phase of the Leonardo da Vinci programme, and its implementation over the first four years. They identify a number of improvements in relation to the programme's first phase, however they also point to a some shortcomings, which impact negatively on the performance of the programme.

5.1. Relevance

External evaluation and national reports agree that the Leonardo da Vinci programme is a highly relevant programme. The common view is that the programme is consistent with the recent political developments and initiatives in the domain of vocational education and training, and addresses the needs of its target groups. "In general, the objectives and priorities of the Leonardo da Vinci II programme appear to meet the main expectations and requirements of the actual promoters and beneficiaries of the programme, judging by the high degree of satisfaction noted during the survey which we carried out of the actual promoters and beneficiaries of the programme, and the views expressed by the national agencies." (external evaluation)

The external evaluation however questions the relevance of certain measures - defined in the Decision - for being properly adapted to the programme's objectives and priorities. The 'mobility' and 'pilot project' measures with more than 80% of the budget during the first 4 years are considered as highly relevant for the achievement of the objectives. The measures, which have access only to a 'small' percentage of the overall budget, are considered as less relevant in this context.

5.2. Efficiency of the programme

At this stage of the programme, when only a small number of projects have been fully completed, an efficiency assessment can mainly be made in relation to the administration costs/effectiveness ratio of the programme.

"This ratio appears to have improved significantly compared with the previous phase of the programme. " (external evaluation)

However it appears that the efficiency of the Leonardo programme can be further improved. In particular the absence of efficient tools for collecting information on an on-going basis, a number of rather 'heavy' and time consuming administrative procedures and reporting mechanisms provide room for improvement.

The Commission services are developing a new tool 'Symmetry', which should make it possible to resolve by 2005 a good number of the identified shortcomings and helping to stream-line the reporting and feedback mechanisms. Its implementation should start at the end of 2004.

5.3. Effectiveness of the programme

The national reports and the external evaluation consider Leonardo da Vinci as an effective programme. According to the external evaluation, "Overall, the Leonardo da Vinci programme is quite effective and has improved over the period 2000-2002, although disparities exist depending on the objectives involved."

On an operational level, the effectiveness of the programme varies, depending on the measure concerned. The measures that use the major part of the budget (almost 85% - mobility and pilot projects) are considered highly effective, whereas the other measures with much less funding seem to be less effective. The external report underlines that accomplishment of all of the operational objectives between now and the end of the programme is possible.

In view of the expected results of the programme (quality of projects) and its contributions to the achievement of its specific objectives, the external evaluation considers the overall effectiveness as satisfactory. It expects that the projects will deliver quality results, not only because the quality of proposals has improved but also because of the close follow-up and monitoring of the project content by the National Agencies and the thematic monitoring activities that are co-ordinated by the Commission services.

The external evaluation concludes that "Most importantly, the overall effectiveness of the programme has improved in relation to the previous decade."

At this stage of the implementation of the programme, it is premature to measure the impact of the programme. Nevertheless, the external evaluation and the national reports are optimistic vis-à-vis the 'anticipated impact' of the programme.

The external evaluation states: "Thus, the Leonardo programme is perceived by all players concerned as a programme with a strong value added component in terms of the development of Community-level co-operation in the area of vocational training".

ANNEXES

Annex 1: Overall Budget of the programme for the years 2000-2003

Annex 2: Procedure A "Mobility" - 2000-2003

Annex 3: Procedures B and C - years 2000, 2001, 2002 and 2003

>TABLE POSITION>

>REFERENCE TO A GRAPHIC>

>REFERENCE TO A GRAPHIC>

>REFERENCE TO A GRAPHIC>

>REFERENCE TO A GRAPHIC>

>REFERENCE TO A GRAPHIC>