52012DC0588

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL AND THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE

on the European Training Foundation

Introduction

This report is being presented under Article 24 of the Council Regulation establishing the European Training Foundation (hereafter ETF) which requires that the Commission conducts every four years an evaluation of the implementation of the Council Regulation, the results obtained by the Foundation and its working methods and that the Commission presents the results of the evaluation to the European Parliament, the Council and the European Economic and Social Committee. The last external evaluation was completed in 2006 and resulted in a Commission Communication to the European Parliament, the Council and the European Economic and Social Committee[1].

The present report provides an overview of experience acquired by the work of the Foundation between 2006 and 2010[2]. It takes into account changes in the role, the geographical coverage, the European Union's external relations environment and the activities of the Foundation since 2006. As its basis, the report draws on the findings, conclusions and recommendations of the independent evaluator's report provided by Public Policy and Management Institute - (PPMI), the external contractor (see section 2). The Commission's experience in activities and cooperation with the Foundation has also been taken into account, as have the recommendations made by the Commission in its 2006 Communication. It should be noted that the report has been drawn up jointly by all services of the Commission working with the Foundation, namely DGs Education and Culture (as the responsible DG), Enlargement, and Development and Cooperation, and in consultation with all services of the Commission working with the Foundation as well as with the European External Action Service.

This report addresses the following topics:

1.           Evolution of the Foundation from 2006 to 2010

2.           Overview of the external evaluation process

3.           Main recommendations of the external evaluation report

3.1     Organisational, thematic and procedural changes

3.2     Relevance

3.3     Coherence

3.4     Effectiveness

3.5     Impact and added value

3.6     Efficiency and cost effectiveness

4.           Conclusion

1.           Evolution of the Foundation from 2006 to 2010

1.1.        Legal basis

The Foundation is a European Union Agency legally established by Council Regulation (EEC) no. 1360/90 of 7 May 1990 establishing a European Training Foundation, which entered into force in October 1993 when the European Council decided on Turin as the seat of the Foundation. Its mission is to help transition and developing countries to harness the potential of their human capital through the reform of education, training and labour market systems in the context of the EU's external relations policy. In 2008, the Foundation's mandate was reformulated in a holistic manner to encompass a broader perspective of human capital development and lifelong learning. Therefore, the EU regulation governing the ETF was recast[3], creating a new mandate for the ETF to address human capital development issues, which widened its formal scope beyond vocational education and training to include issues such as labour market needs and enterprise development. In addition, the new mandate allowed for the geographic scope of the ETF to be expanded if necessary. The new regulation did not include any programme management functions for the ETF, consolidating its role as a centre of policy expertise in human capital development for EU external policies. As a result, the ETF undertook significant reform of its internal structures starting in 2008. Most recently, the Operations Department was split into three separate departments – the Thematic Expertise Development Department, the Geographical Operations Department and the Evidence-based Policy Making Department.

1.2.        Subsidy

A key part of evolution of the Foundation from 1997 to the present has been the level and nature of its subsidy. This comes from the external relations chapter of the European Union budget and, since 2000, has been split between two budget lines - one from DG External Relations (B7-664) and one from DG Enlargement (B7-033). The subsidy drawn from B7-664 is itself drawn from the budgetary envelopes of the 3 external relations programmes relating to the geographical regions of the Foundation's mandate, namely the Western Balkans, Eastern Europe and Central Asia and the Mediterranean.

The level of subsidy for each year in the period 2000-2004 was planned in November 2000 under the Governing Board agreement on the mid-term perspectives, subject to the annual budgetary procedure. This subsidy, constituting almost exclusively the Foundation's revenue, serves to fund the Foundation's personnel (Title 1), its infrastructure and running costs and overheads ensuing from the execution of statutory tasks (Title 2) and operational expenditure relating to its specific missions (Title 3).

The table below shows the evolution of the subsidy from 1998-2012 including the modification of the source of the subsidy away from Phare towards the other regions under the external relations framework and the inclusion in title 15 Education and culture (from 2006 onwards) and under Heading 4 European Union as a Global Partner from 2008 onwards. Annexes 2 and 3 show the full budget evolution (including funds committed, carried over and unused) and the Activity Based-Budgets 2000-2004, to illustrate in a more representative way how funds have been allocated.

While for the years 1998 to 2007 the budget is divided into two lines corresponding to external relations policies and instruments (in summary, Relex and Enlargement, despite there being a change in the budget line code for 2006 and 2007) from 2008 onwards the two lines are directly related to the ETF distribution by titles (15 02 27 01 for title 1 and 2 and 15 02 27 02 for title 3). From 2006 onwards the subsidy budget line code starts with 15 xx, which corresponds to Education and Culture. As concerns the staff, the gradual reduction in the number of temporary agents in the Establishment Plan, since 2006, has been compensated with an increased number of Contract Agents. The total number of posts has remained stable at 135.

Year || European Union subsidy || Number of staff temporary agents

1998 || 15.4m || 119

1999 || 16.2m || 124

2000 || 16.2m || 119

2001 || 16.8m B7-664: 9.24m B7-033: 7.56m || 115

2002 || 16.8m B7-664: 12.8m B7-033: 4.0m || 105

2003 || 17.2m B7-664: 13.7m В7-033: 3.5m || 104

2004 || 17.6m B7-664: 15.1m B7-033: 2.5m || 104

2005 || 18.5m B7-664: 16m B7-033: 2.5m || 104

2006 || 19.45m 15-03-02: 2.95m 15-03-03: 16.5m || 105

2007 || 19.7m 15-03-27-01: 15.6m 15-03-27-02: 4.1m || 100

2008 || 17.98m 15-02-27-01: 14.59m 15-02-27-02: 3.39m || 96

2009 || 14.77m + 5.10m = 19.87m 15-02-27-01: 15.57m 15-02-27-02: 4.30m || 96

2010 || 18.28m + 0.71m = 19.99m 15-02-27-01: 15.93m 15-02-27-02: 4.06m || 96

2011 || 20.35m 15-02-27-01: 14.33m 15-02-27-02: 6.02m || 96

2012 || 20.04m 15-02-27-01: 15.08m 15-02-27-02: 4.96m || 96

2.           Overview of the external evaluation Process

As stipulated by Article 24 of the founding Regulation (recast) of 2008, the Commission's procedure of monitoring and evaluation of the Foundation should be carried out with the help of external experts, with the first results of this procedure to be submitted in a report to the European Parliament, the Council, and the European Economic and Social Committee. In March 2011, DG Education and Culture contracted PPMI to carry out an independent external evaluation of ETF. This was based on terms of reference agreed between the Commission services and with the Governing Board. A Steering Committee was established, chaired by DG Education and Culture with representatives from DGs Education and Culture, External Relations, Enterprise, Europe Aid, and the Foundation.

The external evaluator used several methodological tools. Three separate but complementary surveys were carried out, focusing on two groups of beneficiaries (partner countries and the European External Action Service/European Commission) and a survey of ETF operations expert personnel that provide services to beneficiaries. Included in the surveys was a question asking respondents to identify their most important contacts in the field of human capital development. With these data, social network analysis was used to construct a network of actors involved in the field of human capital development at the European and partner country level in order to provide insight into the nature of relations between the ETF and its beneficiaries. Interviews were carried out in partner countries, the ETF, the European Commission and the European External Action Service. Moreover, in-depth case studies were undertaken in four countries representing the four regions targeted by different EU external policy instruments and ETF actions. Finally, a modified cost-effectiveness analysis was used that focused on the immediate ETF outputs and their costs obtained from corporate performance data.

The external evaluator submitted three reports to the Commission: their inception report was accepted on 29 June 2011, their interim report on 3 October 2011 and their final report on 8 February 2012. The Governing Board discussed the final report, focusing in particular on the executive summary, at its meeting of June 2012.

3.           Main recommendations of the external evaluation report

The Commission invites the ETF to analyse and act upon all of the evaluator's recommendations. To contribute to this, the Commission summarises and analyses below those findings, conclusions and recommendations of the evaluator which, in its view, deserve highlighting. In addition, the Commission gives, where appropriate, specific details or proposals for implementing the recommendations.

3.1.        Organisational, thematic and procedural changes

The evaluator underlined that the ETF underwent significant change during the last five years. Since the 2008 recast of the ETF mandate, there has been significant organisational and operational reform. Key among these changes is the introduction of the Torino process whereby the ETF aims to more accurately and easily identify, enumerate and address human capital development issues in partner countries and develop their capacities for evidence-based policy making. Even though only one year's reports were available for this evaluation, the evaluators viewed the Torino process outcomes favourably.

3.2.        Relevance

The evaluator found ETF's to be relevant both thematically and procedurally to beneficiary needs and that it managed to strike a good balance between flexibility and strategic planning. However, the evaluator stressed that the ETF should be more proactive in clarifying its role to stakeholders with whom they work and the ways in which the ETF can provide support. The ETF has performed very well in developing human capital development policy but beneficiaries are not always aware of this impact and should be made aware of ETF's success and impact, e.g. a clear description of areas in which ETF can provide support.

Moreover, it is noted that the ETF role is clearly defined at the policy level and with state level actors. However, the ETF role vis-à-vis social partners and NGO need to be clarified.

3.3.        Coherence

The evaluators found ETF actions to be internally coherent, as well as being coherent with broader EU objectives, notably in EU external relations, and most partner country objectives. However, it was found that there was no clear hierarchy between strategic and operational ETF objectives in mid-term perspectives and annual work plans.

Therefore, it is recommended that the annual work programmes and country plans identify specific areas of action (both thematically and procedurally) and clearly state how these actions contribute to wider strategic objectives of the ETF.

3.4.        Effectiveness

The evaluator found ETF to be highly effective in its main roles of providing and disseminating information, analysis, and policy advice, capacity building and networking and knowledge transfer.

Information dissemination has evolved over the evaluation period and is now making greater use of social media and online opportunities to share information. However not all stakeholder groups are clear on the nature of ETF information and action. Therefore, in particular with stakeholder groups more recently engaged by the ETF (i.e. social partners and NGO's) a more proactive approach to information dissemination should be undertaken in order to inform these groups about the nature and extent of ETF action.

The ETF was effective in helping to develop knowledge and expertise which could lead to greater partner country capacity in human capital development. However, it was difficult to distinguish how effective the ETF was in building capacity, as a picture of how ETF actions should contribute to capacity building and a clear definition of capacity building were not established. It is therefore recommended for ETF to continue to work closely with partner countries to develop a strategy for how ETF actions and knowledge development can be utilized to improve capacity and increase partner country ownership over the policy process.

Finally, the evaluators stressed that the ETF performed most strongly in networking and knowledge transfer and that these activities were highly valued by beneficiaries. Regional networks were especially valued. In that context, the reorganization and the creation of separate thematic and geographic departments offers potential to strengthen further regional initiatives in terms of networking and policy learning between stakeholders. These regional networks should continue to be developed in the future to the greatest extent possible.

3.5.        Impact and Added Value

The evaluator found ETF to perform very well, in particular given its wide mandate and limited resources. It is recommended for ETF to maintain consistency at the partner country level by matching staff skills with particular thematic and geographic areas. Country managers should therefore be rotated infrequently in order to allow them to build up sufficient knowledge and connections at the partner country level.

3.6.        Efficiency and Cost Effectiveness

The evaluators found ETF to have a system of useful performance indicators which should be kept stable to enable measuring the progress of the organization over the years. However, the existing indicators focus solely on immediate outputs. The indicators should therefore be extended to incorporate results level indicators, e.g. partner country beneficiary satisfaction with services provided by ETF.

It was found that ETF is cost-effective in its work. Given its vast thematic mandate, large geographical area and relatively modest budget, the ETF has proven to be flexible in the past in deploying its support where EU institutions and Governing Board deemed it most necessary.

4.           Conclusion

The Commission considers that within the Union's external relations policy framework, the ETF has a useful contribution to make as a centre of expertise in all four regions of its mandate.

The Commission considers that the ETF has indeed been able to successfully reorganize and refocus its activities. The Commission shares the evaluator's overall positive assessment of the effectiveness of the Foundation's work in its main roles of providing and disseminating information, analysis, and policy advice, capacity building and networking and knowledge transfer. The ETF has also taken up the challenge of its recast mandate for considerable internal reforms. However, it is recommended for ETF to be more proactive in disseminating information to relatively new stakeholders. In the field of capacity building, the ETF should work closely with partner countries to develop a strategy for how ETF actions and knowledge development can be utilized to improve capacity and increase partner country ownership over the policy process. Finally ETF's focus on the development of regional networks should be further strengthened.

Moreover, the Commission stresses the relevance of ETF's activities to beneficiary needs but agrees with the evaluator that a more proactive approach in clarifying ETF's role to stakeholders is wanted as well as a clearer definition of its role vis-à-vis social partners and NGOs. While ETF's actions were found to be coherent, the Commission agrees with the evaluators in underlining the need to explain in the annual work programmes and country plans how specific actions contribute to wider strategic objectives of the ETF.

Finally the Commission shares the evaluators' positive assessment of ETF's added value and cost-effectiveness, in particular given its wide mandate and extensive geographic coverage.

[1]               COM(2006) 832 fina 19.12.2006

[2]               The evaluation process was launched at the end of 2010 and the final report was issued on 8/02/2012; it can be found at http://ec.europa.eu/dgs/education_culture/evalreports/training/2012/etf_en.pdf

[3]               Regulation (EC) No 1339/2008 of the European Parliament and of the Council of 16 December 2008 establishing a European Training Foundation (recast) – OJ L 354, 31.12.2008, p. 82