52006DC0832

[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |

Brussels, 19.12.2006

COM(2006) 832 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL AND THE ECONOMIC AND SOCIAL COMMITTEE

European Training Foundation

1. INTRODUCTION

This report is presented under Article 17 of the Council Regulation establishing the European Training Foundation (hereafter the ETF), modified by the Council regulation of 2003[1] which requires that the Commission creates a monitoring and evaluation procedure for the ETF and reports on its results.

This report aims to provide an overview of experience acquired in the work of the ETF for the period 2002-2005. It takes into account changes in the geographical coverage, operating environment and role of the ETF since 2000, as reflected in the Mid-term perspective on staff and financial resources adopted by the ETF Governing Board in November 2000 and in the Mid-term Perspective 2004-2006 adopted by the ETF Governing Board in November 2003.

The report draws on the findings, conclusions and recommendations of the independent evaluator's report provided by ITAD, the external contractor (see section 3). The Commission's experience in activities and cooperation with the ETF has also been taken into account, as have recommendations made by the Commission in its 2003 Communication.

The report also takes into consideration the evolution of EU policy towards third countries, and expected changes in EU policy instruments after 2006. The current external assistance programmes (Phare, Tacis, Meda and Cards), which define the scope of the ETF’s activities, will be replaced in 2007 by three new external assistance instruments, the European Neighbourhood and Partnership Instrument (ENPI), the Instrument for Pre–accession Assistance (IPA) and the Development Cooperation Instrument (DCI). These changes require an adaptation of the ETF’s founding regulation. The findings from the external evaluation process will inform the proposed amendment of the Council Regulation on the ETF.

The report has been drawn up jointly by all services of the Commission working with the ETF, namely DGs Education and Culture (the responsible DG), External Relations, Enlargement, EuropeAid and Employment, Social Affairs and Equal Opportunities.

2. EVOLUTION OF THE EUROPEAN TRAINING FOUNDATION, 2002-2005

2.1. Background to ETF

The ETF[2] is located in Turin, Italy, and has 105 staff. It is a Community Agency legally established by Council regulation (EEC) no 1360/90 of 7 May 1990. The Agency's overall objective, as described by Articles 1 and 2 of the Regulation, is to contribute to the development of vocational training systems in the four regions within its remit: the Caucuses and Central Asia[3], the acceding and candidate countries[4], the potential candidate countries[5], and the southern and eastern Mediterranean[6]. Beneficiary countries are known as "partner countries".

The ETF now provides services to six Directorates-General (DGs) of the European Commission: Education & Culture (the ETF's supervisory DG), External Relations, EuropeAid, Enlargement, Enterprise (regarding the SME Development Fund), and Employment, Social Affairs and Equal Opportunities.

In terms of service provision, the ETF's principal interlocutors in the candidate and potential candidate countries tend to be the EC delegations (with the European Agency for Reconstruction[7] the main interlocutor for Serbia, Montenegro and the former Yugoslav Republic of Macedonia). For the other areas (Caucasus, Central Asia and Neighbourhood), it is EuropeAid, which is responsible for the implementation of EC cooperation activities together with the relevant EC delegations.

2.2. Current priorities and activities

ETF current priorities are derived from the goals set in its Mid-term Perspective 2004-2006 adopted by ETF Governing Board in November 2003. The operational priorities are based on EU external relations policies towards each partner region.

On an operational level, the ETF uses three main instruments:

- Support to EC programmes and dissemination of EU policies.

- Capacity building, information and analysis for the partner countries.

- Innovation, Learning and Development, including pilot projects which analyse lessons learned during the application of reform strategies by partner countries.

2.3. Previous evaluations, studies and reviews

An external evaluation was carried out in 2002[8] and led to a Commission Communication in mid-2003[9]. A Communication of 1997[10] reported on the findings of a first external evaluation.

2.4. Funding

ETF funding comes from the external relations chapter of the Community budget and, since 2000, has been split between two budget lines - one from DG External Relations (15 03 03; ex-B7-664) and one from DG Enlargement (15 03 02; ex-B7-033).

The level of funding for each year in the period 2002-2005 has been planned under the Governing Board agreements on rolling mid-term perspectives (2002-2004; and subsequently, 2004-2006). These planning forecasts are subject to the annual budgetary procedure. This funding constitutes almost the entirety of the ETF's revenue.

Annex 2 shows the full budget evolution (including funds committed, carried over and unused) to illustrate in a representative way how funds have been allocated.

3. OVERVIEW OF EXTERNAL EVALUATION EXERCISE

Article 17 of the founding Regulation, modified by the Council Regulation of 2003, stipulates that the Commission should evaluate ETF every three years. The current external evaluation was launched in late 2004 with a report due in 2006.

DG Education and Culture launched an open call for tender for an independent external evaluation of the agency. The contract was awarded to ITAD Ltd (the same organisation that carried out the previous evaluation). A Steering Committee was established, chaired by DG Education and Culture with representatives from DGs External Relations, Enlargement, EuropeAid, Employment, three Member States (Governing Board members) and the ETF.

The overall objectives of the evaluation were:

- To assess the extent to which the commitments made by the ETF in its 2002, 2003 and 2004 Work Programmes and in the 2003 Action Plan were achieved;

- To assess the extent to which the commitments made by the Commission in its 2003 Communication were met;

- The provision of useful lessons and recommendations for the challenges facing the ETF in the programming phase 2006-2008.

The external evaluator used several methodological tools. A detailed matrix was drawn up in order to breakdown the main evaluation questions summarised above and in order for the Steering Committee to see exactly how each question was to be answered. The evaluators reviewed a significant number of documents relating to the agency's activities and its own evaluations over the three-year period. Interviews were carried out with Commission staff, with MEPs and with the majority of ETF’s staff. Complementing this qualitative information were two surveys: one aimed at stakeholders in the Commission, in the European Parliament, in the Member States, in partner countries and in other bodies; and a separate survey of ETF’s staff. In addition, four case studies that represented the different regions, types of activity, thematic area and period of implementation were selected and visits made to four partner countries to explore these cases in depth.

Initial findings were presented to the Governing Board during its meeting in November 2005; the overall conclusions and recommendations from the external evaluation process were also discussed at its meeting of 6 June 2006.

4. MAIN RECOMMENDATIONS OF THE EXTERNAL EVALUATION REPORT WITH COMMISSION'S REACTION AND RECOMMENDATIONS

In this section the Commission summarises and analyses those findings, conclusions and recommendations of the evaluator which, in its view, deserve highlighting. Certain recommendations ask for action by the Commission, others are targeted to the ETF. The Commission invites the ETF to analyse and act upon the evaluator’s recommendations and to identify the actions it intends to take. A follow-up of the findings will be organised at the agency level (via the Governing Board) and on Commission level.

The Commission will pay particular attention to the recommendations made in the external evaluation report when drafting the amendment of the ETF’s Council Regulation.

To facilitate cross-reading with the external evaluation report[11], this section follows the same structure as the ‘Recommendations section’ in that report.

4.1. Policy and Strategy

The evaluators found that ETF delivers a focused set of services with clear policy objectives, articulated in the Mid Term Perspective and Work Programme Planning documents, which vary by region according to EU external relations policy priorities. Furthermore, the evaluation found that the ETF’s programmes were in line with both EU and individual country strategies and that the ETF has been able to influence the direction the reform might take.

The evaluation confirms that the work provided by ETF is seen as good value. The Commission in Brussels and the Delegations have a positive perception of ETF’s provision of expertise in VET across a range of services. ETF added value derives from the sustainability of the institution, its understanding of the reform context, its network of experts and its capacity to respond flexibly and rapidly to requests.

The evaluation concluded that ETF’s current thematic mandate limiting it to VET is too restrictive, while appreciating that the mandate has been interpreted over the last years with some flexibility by both the Commission and ETF. The upcoming revision of the ETF’s mandate should “ take account of the view that VET is one part of a wider programme of Human resource Development (HRD)” and the evaluators recommend that the planned revision “ should accommodate the terms Human Resource Development (HRD), VET, Life Long learning and links to the labour market and employment ”.

The report also argued for some increased flexibility in the ETF’s geographical remit. This would allow ETF’s particular expertise to be used by the Community in countries at comparable stages of development and facing similar socio-economic challenges. The evaluators underline that any greater flexibility in interpreting the mandate should be accompanied by the establishment of mechanisms to ensure a rigorous prioritisation of ETF’s activities.

The Commission shares to a large extent the analysis and recommendations of the evaluators, in particular on the thematic remit of the agency. The growing importance of the lifelong learning approach to education and training in the policies and instruments of the EU calls for a wider thematic remit of the ETF. The Commission proposes to address this issue in the upcoming revision of the ETF Regulation by broadening its mandate to ‘contribute, in the context of EU external relations policies, to the improvement of human resources development, in particular education and training in a lifelong learning perspective, and related labour-market issues’.

The Commission considers that the ETF, which has limited resources, should maintain its geographical focus. It is therefore proposed that the ETF’s primary scope be defined by reference to the Instrument for Pre-Accession (IPA) and to the European Neighbourhood and Partnership Instrument (ENPI). In order to ensure the possibility to use the ETF’s expertise in areas falling outside its primary geographical scope, such as the Central Asian countries, it is proposed that a relevant ad hoc decision be taken by the ETF Governing Board on proposal from the Commission.

4.2. Efficiency and effectiveness of ETF programmes

The report assessed the efficiency and effectiveness of ETF’s activities particularly from the point of view of its work at partner country level. The main source of information and analysis were the country studies which look in depth at the efficiency and effectiveness of ETF activities in a sample of four partner countries. The evaluators found it challenging to assess the specific results and impacts achieved by ETF given the nature of the sector, the scale of ETF’s activities and the range of different stakeholders.

In general, the report concluded that ETF had been efficient and effective in its contribution to VET reform in the partner countries. “ From the perspective of both the Commission Services in Brussels and the Delegations, the contribution of ETF is perceived as very positive as it brings value to its users through provision of expertise in VET and Human Resource Development (HRD) across a range of services.”

Equally, ETF’s support to the dissemination of experiences in VET policy and reform was seen as a useful contribution to the reform process, as well as increasing the familiarisation of stakeholders in the countries with EU policy approaches. The evaluators found that ETF’s own activities complemented Commission funded programmes, and contributed to greater efficiency in their implementation. A key issue arising from the report, however, is to ensure a close link between ETF pilot actions and larger scale EC funded actions, or actions which could be funded by other donors.

In conclusion, the report argued that ETF should provide a mix of different services both to the Commission and to partner countries.

Based on this analysis and in view of the new external assistance instruments, the Commission proposes to review the functions of the ETF in the up-coming revision of the agency’s regulation. The Commission considers that, in the coming years, the ETF should have the following functions:

- Provide information, policy analysis and advice on human resources development issues and their links to sector policy objectives in partner countries;

- Support stakeholders in partner countries to build capacity in human resources development

- Facilitate the exchange of information and experience among donors engaged in human resource development reform in partner countries;

- Support the delivery, follow-up and analysis of Community assistance to partner countries in the field of human resources development;

- Disseminate information and encourage networking and exchanges of experience and good practice between the European Union and partner countries and amongst partner countries on human resources development issues.

4.3. Monitoring of ETF programmes

The evaluation report paid attention to the need to raise the ETF's capacity to monitor its activities at project and country level. In this respect, the report recognised that the monitoring of ETF achievements at country level is a complex area in that much of the ETF’s work cannot be simply “project-ised”; as a set of activities that contributes to specific outputs and results. The ETF’s work is more about facilitating the flow of information rather than about producing products or outputs (publications, new curricula, etc). Also, in many cases, the ETF had essentially acted as a service provider for the EC services in supporting the definition, set-up and follow-up of projects.

The report states that the main challenge for the ETF is to monitor not simply the production of a particular document, but to measure the impact produced on national plans and reform actions. The evaluation concluded that the ETF’s current monitoring system, which is designed to track performance at the level of activities and individual projects, did not adequately measure the quality of the ETF's services/products or the ETF's impact on VET reform at country level.

The Commission agrees with the positive conclusions of the evaluators on the efficiency and effectiveness of the ETF’s technical assistance to the Tempus programme. It appreciates in particular that the ETF's Tempus Department continued to respond to requests for changes in the way it works and continued to be organised efficiently in spite of the uncertainties over the future of Tempus which characterized the period of the evaluation.

The Commission shares the analysis and conclusions of the external evaluator that ETF needs to reinforce its monitoring system on country level. Available country programme information needs to be collected in a systematic way in order to inform adequately decisions at the organisational level. The Commission asks the ETF to consider the criticism and recommendations made by the external evaluator and to develop a relevant monitoring system to address identified shortcomings. As the evaluator states, “ the challenge is to develop an approach that provides meaningful information at this level, and in a way that is not cumbersome and mechanistic .”

4.4. Governance structures

The evaluators looked in particular at the role of the Governing Board and the issue of providing strategic direction; membership qualifications; conduct of meetings; new member induction process, and information and briefing materials provided ahead of meetings.

The report notes that the primary role of the Governing Board is inter alia to approve the proposed annual work programme and budget, subject to final approval by the European Parliament. The evaluators conclude that, although the Board does fulfil its primary role, there are questions regarding its effective role in strategic decision making. The report recognises that the main strategic direction comes from the Chair and Commission Members; it questions to what extent it is realistic to expect other members to contribute meaningfully to the strategic vision, pointing to a lack of alignment between the largely internal EU education policy background of Board Members and the predominantly external relations policy role and responsibilities of the ETF.

On the Advisory Forum, the report concluded that the network provides a good opportunity for dialogue between representatives of the European Commission, Member States, partner countries and international organisations. However the evaluation stated that the Advisory Forum falls short of its purpose of providing advice to the Governing Body on the future Work Programme.

The Commission shares to a certain extent the analysis and recommendations made the by external evaluator. However, ETF’s objectives are defined in its legal basis and its activities and priorities are to a large extent determined by the external policies and instruments of the EU. Therefore it is questionable how strong the strategic role of the Governing Board could be and/or should be. The recommendation to establish sub-committees on specific important files needs further analysis. The Commission is not convinced by the proposal to seek funding for a post for a desk officer at the ETF to support the Governing Board.

The Commission has submitted to the European Parliament and to the Council a draft Inter-institutional agreement on the operating framework for the European regulatory agencies (COM(2005)59; Feb 2005) (IIAOFERA). That agreement proposes that the size of the administrative boards of Agencies be limited in such a way as to promote highly effective decision-making and minimise operating costs, and that there should be parity in the number of votes between the members designated by the Council and those designated by the Commission. The Commission in its proposal for a revised regulation of the ETF will follow these principles.

The Commission shares the conclusion of the evaluator that “ the Advisory Forum is de-linked from its statutory function ”. While the Advisory Forum is an important network, it has largely lost its statutory function as an organ of governance of the ETF. The programming of ETF’s activities is increasingly decided by a process of dialogue with the Commission on the basis of EU external assistance priorities, and the statutory opinions of the Advisory Forum no longer play a significant role in determining the content of the ETF’s activities on an annual basis. It is therefore proposed not to include it as a statutory body in the revision of the ETF’s regulation. The Commission calls on the ETF to analyse the evaluator's recommendations when reviewing the Advisory Forum.

4.5. Communications and information dissemination

The assessment concluded that ETF has a successful and professional communications system in operation at corporate level, through its website and publications. The evaluators reported that the communication and dissemination system had greatly improved since the last evaluation. The report showed that ETF’s communications initiatives, and in particular its website, workshops and exchange visits, newsletter printed publications (80%) were very effective.

However, the report stated that there might be an overemphasis on promoting ETF as an institution. The report suggested therefore that ETF should reinforce its capacity to efficiently communicate its achievements and results as well as to disseminate ideas and material on thematic areas relating to VET. It should also consider broadening the dissemination of materials in partner country languages to increase accessibility of its work to partner country stakeholders.

The Commission welcomes the positive developments in the field of communication and dissemination achieved since the last evaluation. It asks the ETF to analyse the recommendations made by the external evaluator and to propose relevant remedial actions.

5. CONCLUSION

The Commission shares the evaluator's overall positive assessment of the efficiency and effectiveness of the ETF's work and considers that the Agency has made a valuable contribution to the Community's activities in the field of vocational education and training and labour market reform in the partner regions. The Commission appreciates the ETF’s work in providing expert input for the identification and implementation of projects.

As highlighted by the evaluators, the ETF has to look at its planning cycle, in particular the it is asked to analyse how the links between ‘the mid-term priorities – the annual work programme – the country plans – the annual activity report’ can be reinforced and be made more explicit. Relevant measurable indicators should be defined in order to facilitate the follow-up of the set objectives and goals and to ease the assessment of the ETF's impact in its areas of intervention. New approaches should be explored to improve the development and appropriate dissemination of relevant VET material, and to strengthen the ETF’s capacity to communicate its achievements and results.

The introduction of the new external aid instruments, the IPA and the ENPI, marks a significant step towards a policy-driven rather than programme-driven approach to external assistance in a sector-wide perspective. Both instruments are designed to provide support towards the achievement by partner countries of home-grown reforms in the sectors concerned. The objectives are defined by a process of dialogue with partner country governments and support is provided to them for the implementation of reform processes designed and managed by them.

In this new context, the ETF will increasingly be required to provide the Commission with information and analysis at the policy development and programming phase. The other focus of attention has to be in helping partner countries to build the capacity to define and implement indigenous reform strategies, and in promoting networking and exchange of experience and good practice between the EU and the partner countries and among partner countries themselves.

The ETF should adapt its organisation and its way of working to this new environment. The agency must in coordination with the Governing Board and the Commission establish clear priorities and concentrate its available resources on its core activities.

For both the Commission and the ETF, increased awareness and monitoring of the type of services both offered and needed will be required in order to ensure that the activities of the Agency continue to be useful and relevant. This implies that the Commission should keep the ETF fully informed on new or revised priorities or strategies. Similarly, the ETF should keep Commission services, in the field and at headquarters, fully informed of its operations, initiatives and contacts with relevant stakeholders in partner countries, other international donors and agencies in order to ensure coherence and visibility of EC action. This demands a renewed commitment from both sides to strong communication and exchange.

The ETF's management must continue its effort to establish a clear vision within the Agency of its position and role as a European centre of expertise in the new institutional landscape.

Overall, the Commission believes that the external evaluation report provides useful lessons and recommendations for the development of the ETF as a centre of expertise. The Commission invites the ETF to submit a draft Action Plan to the Governing Board, outlining its analysis and proposed actions on all recommendations made by the evaluator and by the Commission in this report.

The Commission proposal for a revised regulation of ETF will also provide the opportunity for discussion and exchange with the European Parliament and the Council on the future role of the ETF, its objectives and priorities, and its governance structure.

[pic]

Annex 2

BUDGET EVOLUTION, 2002-2006

Year |Community subsidy | Staff costs | Administration and infrastructure costs |Operational costs |Establishment Plan | Funds committed |Funds carried over to following year |Unused funds | |2002 | B7-664: 12.8

B7-033 4.0

Total: 16.8m |9 890 000

(63.0%)

|1 386 050

(8.0%)

|5 523 950

(29.0%) |105 |16 795 986 |3 365 833 |4 014 | | 2003 | 15 03 03 13.7

15 03 02 3.5

Total: 17.2m |10 529 000

(65.0%)

|1 421 000

(8.0%) |5 250 000

(27.0%) |104 |16 969 378 |1 725 970 |230 622 | | 2004 | 15 03 03: 15.1

15 03 02: 2.5

Total: 17.6m | 10 993 973

(62.0%)

|1 390 027

(8.0%)

|5 276 000

(30.0%) |104 |17 321 617 |1 537 855 |278 383 | | 2005 | 15 03 03: 16.0

15 03 02: 2.5

Total: 18.5m | 11 287 000

(61.0%)

|1 453 000

(8.0%)

|5 760 000

(31.0%) |104 |18 089 560 |2 729 149 |410 440 | | 2006 | 15 03 03: 16.95

15 03 02 : 2.5

Total: 19.45m | 11 956 800

61.0%

|1 688 000

9.0%

|5 805 200

30.0% |105 | | | | |Since move to Activity Based Budgeting in 2003: 15 03 02 replaces B7-033; 15 03 03 replaces B7-664

[1] Council Regulation EC N° 1648/2003 of 18 June 2003

[2] ETF website: http://www.etf.eu.int/

[3] Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Mongolia, Russian Federation, Tajikistan, Turkmenistan, Ukraine, Uzbekistan

[4] Bulgaria, Croatia, Romania, Turkey and the Former Yugoslav Republic of Macedonia

[5] Albania, Bosnia & Herzegovina, Serbia including Kosovo (under UN Security Council Resolution 1244), and Montenegro

[6] Algeria, Egypt, Israel, Jordan, Lebanon, Morocco, Palestinian Authority (West Bank & Gaza Strip), Syria, Tunisia

[7] See EAR website: http://ear.europa.eu/

[8] External Evaluation of the European Training Foundation , ITAD, November 2002: http://ec.europa.eu/dgs/education_culture/evalreports/index_en.htm#ETFinterim1

[9] COM(2003) 287, 22.5.2003

[10] COM(1997) 379, 18.7.1997

[11] External Evaluation of the European Training Foundation , ITAD, June 2006: http://ec.europa.eu/dgs/education_culture/evalreports/index_en.htm#ETFinterim3

Director

Muriel Dunbar

Planning, Monitoring and Evaluation

Peter Greenwood

Tempus

Marleen Voordeckers

Operations Department

Sandra Stefani

External Communication

Bent Sørensen

Administration and Central Services

Olivier Ramsayer

ETF Structure1.1.2006