52008DC0484

[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |

Brussels, 24.7.2008

COM(2008) 484 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Interim evaluation of the implementation of the Public Health Programme (2003-2008)

COMMUNICATION FROM THE COMMISSIO N TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Interim evaluation of the implementation of the Public Health Programme (2003-2008)

(Text with EEA relevance)

INTRODUCTION

The Decision of the European Parliament and of the Council adopting a programme of Community action in the field of public health (2003-2008)[1], hereinafter referred to as “the Programme Decision”, established a six-year programme on public health to be implemented from 1 January 2003 to 31 December 2008.

The purpose of this communication is to present the conclusions of the evaluation of the implementation of the first three years of the Public Health Programme (2003-2008), hereinafter referred to as “the programme”.

The need to carry out the evaluation stems from Article 12(3) of the aforesaid Decision, which also requires the European Commission to communicate the results of the evaluation to European Parliament, the Council, the European Economic and Social Committee, and to the Committee of the Regions.

The evaluation was carried out by independent experts from a consultancy company, RAND Europe, selected through a tendering procedure.

The full evaluation report is available online on the DG Sanco Europa website at the following address: http://ec.europa.eu/health/ph_programme/eval2003_2008_en.htm.

2. Background

Under Article 152 of the Treaty establishing the European Community, a high level of human health protection must be ensured in the definition and implementation of all Community policies and activities.

This programme, which complements national policies, is designed to help to achieve the target set in the Treaty.

The general objectives of the programme are to:

- improve health information and knowledge for the development of public health;

- enhance the capability of responding rapidly and in a coordinated fashion to threats to health;

- promote health and prevent disease through action on health determinants across all Community policies and activities.

The programme is thus designed to:

- ensure a high level of human health protection in all Community policies and activities, by way of an integrated and intersectoral health strategy;

- tackle inequalities in health;

- encourage cooperation between Member States in the areas covered by Article 152 of the Treaty.

3. METHODOLOGY

The evaluation was based on several data collection methods, namely desk research, a survey of project leaders, interviews with a variety of stakeholders, analysis of data archives and analysis of the projects database.

It focused mainly on projects co-funded in the period 2003-2005 under the annual calls for proposals, which represented the main activity of the programme. The budget allocated for the calls for proposals accounted for most of the operating budget.

As this was an interim evaluation, it was carried out when most of the projects were at an early stage of development. This meant that the evaluators were unable to measure exactly to what extent the objectives of the programme were met by projects co-funded under the Calls for Proposals for 2003-2005.

Taking these limitations into account, the following issues regarding the Public Health Programme (2003-2008) were addressed in the evaluation.

4. EVALUATION ISSUES AND QUESTIONS

The evaluation assessed the following specific issues:

Relevance : pertinence of the programme objectives to the needs of its stakeholders.

Effectiveness : extent to which the objectives set in the Programme Decision and annual work plans are achieved; degree to which the priorities described in the annual work plans and the Programme Decision have been duly implemented by the programme; extent to which the programme manages to complement national policies (by way of European added value) to protect human health and improve public health at reasonable cost.

Efficiency : timeliness of resource allocation, appropriate quantity, quality and cost-effectiveness; analysis of the programme’s organisational set-up and structure, related responsibilities and overall management capacity in relation to the results achieved; methods used to implement the programme.

Consistency and complementarity : extent to which consistency and complementarity is ensured between measures implemented under the programme and other EU policies and activities (such as regional policy, research, statistical programmes, labour policy, etc.).

Acceptability: extent to which stakeholders accept public health policy in general and the way in which the programme implements this policy.

Monitoring: adequacy of the current monitoring system of programme activities.

Sustainability : extent to which the results of the programme are sustainable in the medium and long term, i.e. after the funding granted by the EU has ceased.

5. FINDINGS

THE KEY FINDINGS of this evaluation are presented below.

5.1. Relevance

A survey of project leaders and interviews with different stakeholder groups has shown that there is a clear perception among stakeholders that the programme delivers benefits for stakeholders and EU citizens alike. Significant benefits mentioned were: network creation, exchange of knowledge and good practices, Europe-wide cooperation and capacity building.

5.2. Effectiveness

The survey showed that projects are considered to support the delivery of the objectives of the Programme Decision and the work plans. “The selection of projects is perceived to have been managed through clear and transparent evaluation procedures published in the Online Journal, reinforcing the sense that only relevant projects have been funded. In this process, evaluators appear to have applied appropriate criteria transparently and this impression is supported by evidence from the surveys and interviews”.[2]

5.3. Efficiency

A significant number of project leaders found the project administration to be complex and bureaucratic, while the project submission procedure was considered to be time-consuming and burdensome.

5.4. Consistency and Complementarity

The extent to which the Public Health Programme (2003-2008) interacts with other Community activities varies by area; in particular, interaction is perceived to be particularly close in the fields of bioterrorism, health systems and pharmaceuticals.

5.5. Acceptability

Interviewees and stakeholders widely accepted and supported European intervention in public health. The added value of the programme derives from the fact that it could be an effective link between the health policies of national governments and the EC, and thus help to support and coordinate the respective measures.

5.6. Monitoring

Of all the programme’s activities, the evaluation focused on the monitoring of projects co-funded under the calls for proposals. The survey and interviews suggested that the monitoring of projects is adequate.

5.7 . Sustainability

Almost half the project leader respondents (47%) thought that the project and its results would be sustainable once financial support for the programme is stopped. Respondents commented that sustainability could be achieved through the use of project outputs; dissemination (e.g. public health reporting); continued collaboration and networking with partners and other organisations; training and additional funding. The evaluation concluded that it is clear that at least some parts of the programme are expected to have sustainable consequences.

6. RECOMMENDATIONS

The overall assessment of the Public Health Programme (2003-2008) is positive. However, eighteen recommendations were issued in three areas: programme, management and projects.

6.1. Programme

The evaluation defined the programme priorities as complicated. Even if the priorities may be entirely justified, they might not reflect the actual needs of stakeholders and/or of EU citizens. Greater attention should be paid to developing sharper priorities through a more needs-driven prioritisation process.

It was recommended that the focus be on areas with high impact and of high concern.

6.2. M anagement

Efforts should be made to develop new funding methods in order to encourage more innovative proposals.

An “inner circle” of stakeholders are very aware of the work of the programme, but there is a wider potential audience that may be being missed. The evaluation suggested that the benefits produced by the co-funded projects should be disseminated through tailored messages to interest groups.

Considering that successful delivery of the programme depends, among other factors, on cooperation with international organisations, it was recommended that existing links with international bodies be maintained and strengthened.

6.3. Projects

It was recommended that the application procedure for submitting projects under the call for proposals be simplified.

The evaluation suggested managing and limiting the number of proposals to be assessed by considering indirect competition, closed competition and informal competition.

The evaluation concluded that projects should monitor their progress against the programme priorities.

7. COMMISSION’S COMMENTS ON THE EVALUTION CONCLUSIONS

7.1. Programme

The evaluation regarded the programme priorities as being complicated. Even if these priorities may have been entirely justified, they might not have reflected the actual needs of stakeholders and/or EU citizens.

Several initiatives have been launched to streamline and prioritise the Public Health Programme activities, such as mapping of the Public Health Programme (2003-2008) and multi-annual planning. These will help to establish priorities for each year of new Health Programme, i.e. the second programme of Community action in the field of health (2008-2013)[3], identify stakeholders’ needs and define specific objectives. This will give the new Health Programme a sharper definition by reducing the number of activities and making priorities more clear-cut.

In addition to this, a broader and innovative consultation process will be pursued in an effort to define the priorities of the annual work plans, address the real needs of EU citizens and increase stakeholders’ awareness of the programme. Several groups of stakeholders will be consulted to identify what for them are the relevant areas of activity.

7.2. Management

As regards the possibility of introducing new funding methods, it should be noted that a wider variety of financing mechanisms are offered in the second Health Programme (2008-2013). These include:

- co-financing an action intended to achieve a programme objective (up to 60% of project costs);

- co-financing the operational costs of a non-governmental organisation or a specialised network (up to 60% of eligible costs);

- joint financing a public body or non-governmental organisation by the Community and one or more Member States (up to 50% of costs);

- joint actions with other Community programmes.

As regards project results, these will be promoted through adequate channels tailored to the different kinds of audience.

Further attention will be paid to project sustainability. Output indicators could be used to ensure closer follow-up of projects. At the end of project implementation, an assessment should be made of the real output produced by the beneficiary. The aim is to check to what extent the final reports reflect expectations at the beginning of the project.

Programme activities will be communicated more widely than in the past. For this purpose, a comprehensive strategy has been developed. New initiatives have been promoted and others which are already in place have been expanded, such as the use of e-newsletters and the web (DG SANCO website, the EU public health portal and the Health-EU e-newsletter, Public Health Executive Agency website), Info Days (based in Luxembourg and National Info Days), further development of the national focal points network, and new health-related publications.

The Commission will step up cooperation with international organisations, such as the WHO and OECD through direct grant agreements.

7.3. P rojects

The application procedure for submitting projects under the call for proposals should be simplified. This has already been done for the 2007 Call for Proposals and will be further pursued under the second Health Programme (2008-2013).

Furthermore, evaluation criteria in use for the call for proposals were revised in 2007. Following the evaluation carried out by the Public Health Executive Agency in 2006, a Commission Decision laying down new general principles and criteria for the selection and funding of actions under the Public Health Programme[4] was adopted on 12 February 2007.

The award criteria were revised with the aim of making the evaluation process more transparent and providing more guidance to applicants on what is expected from projects. The award criteria were clearly detailed and some aspects were made more specific than was previously the case, e.g. what is meant by geographical coverage, the quality and role of partnerships in the context of calls for proposals, European added value, the budget and financial management.

As regards limiting the number of proposals under the calls for proposals through the means proposed by the evaluation, the Commission would point out that the financial regulation prescribes direct competition. Using closed competition in the public health area is not an advantage, and informal competitions do not seem to guarantee impartiality and equal treatment.

Beneficiaries will be invited to describe how their proposals will improve the health of European citizens, as measured where possible by appropriate indicators, including the Healthy Life Years indicator,[5] and how they will reduce health inequalities in and between EU Member States and regions.

8. CONCLUSIONS

This interim evaluation drew largely positive conclusions, while highlighting a number of areas that it recommended for consideration in the further implementation of the Public Health Programme (2003-2008) and for the development of the second Health Programme (2008-2013).

It was an extensive exercise, covering a long period, from the planning stage to finalisation of the report. It ended too late in the policy cycle for any decision to be made on a substantial revision of the Public Health Programme (2003-2008), which was repealed on 1 January 2008 with the entry into force of the second programme of Community action in the field of health (2008-2013).

However, the results of the evaluation were immediately taken into account, whenever possible, in efforts to improve the remainder of the Programme (2003-2008). The Commission will pay the utmost attention to the recommendations of this evaluation during implementation of the second Health Programme (2008-2013).

[1] Decision No 1786/2002/EC of the European Parliament and of the Council of 23 September 2002 adopting a programme of Community action in the field of public health (2003-2008), OJ L 271, 9.10.2002, p. 1.

[2] RAND Interim Evaluation of the Public Health Programme (2003-2008), Final Report, pages 100-101.

[3] Decision No 1350/2007/EC of the European Parliament and of the Council of 23 October 2007 establishing a second programme of Community action in the field of health (2008-13), OJ L 301, 20.11.2007, p. 3.

[4] OJ L 46, 16.2.2007, p. 45-50.

[5] http://ec.europa.eu/health/ph_information/indicators/lifeyears_en.htm