This document is an excerpt from the EUR-Lex website
Document 52007SA0006
Special Report No 6/2007 on the effectiveness of technical assistance in the context of capacity development together with the Commission's replies
Special Report No 6/2007 on the effectiveness of technical assistance in the context of capacity development together with the Commission's replies
Special Report No 6/2007 on the effectiveness of technical assistance in the context of capacity development together with the Commission's replies
IO C 312, 21.12.2007, p. 3–24
(BG, ES, CS, DA, DE, ET, EL, EN, FR, IT, LV, LT, HU, MT, NL, PL, PT, RO, SK, SL, FI, SV)
21.12.2007 |
EN |
Official Journal of the European Union |
C 312/3 |
SPECIAL REPORT No 6/2007
on the effectiveness of technical assistance in the context of capacity development together with the Commission's replies
(pursuant to Article 248(4), second subparagraph, EC)
(2007/C 312/02)
TABLE OF CONTENTS
GLOSSARY
I-VIII |
EXECUTIVE SUMMARY |
1-6 |
INTRODUCTION |
7-8 |
AUDIT SCOPE AND APPROACH |
9-43 |
OBSERVATIONS |
9-12 |
Does the Commission have a sound strategy for institutional capacity development, including the use of technical assistance? |
10-11 |
Inadequate institutional analysis in Country Strategy Papers |
12 |
No guidance on technical assistance |
13-17 |
Are technical assistance activities within capacity development projects well designed? |
14-16 |
Project choice relevant but design often inadequate |
17 |
Local ownership essential but not systematically addressed by the Commission |
18-34 |
Are technical assistance activities implemented efficiently? |
19-21 |
Long preparation phase often leads to outdated project design |
22-26 |
Problems during procurement and project start-up reduce time available for implementation |
27 |
Experts often replaced after award of contract |
28 |
Inappropriate selection criteria for technical assistance |
29-30 |
Limited choice between procurement procedures |
31 |
Inefficient use of technical assistance |
32 |
Donor coordination varies between countries |
33-34 |
Implementation arrangements not yet favorable to local ownership |
35-39 |
Are technical assistance activities and performance adequately monitored and evaluated? |
40-43 |
Are technical assistance activities effective in terms of capacity development? |
40-42 |
Technical assistance contribution satisfactory but success of projects not guaranteed |
43 |
Sustainability uncertain |
44-48 |
CONCLUSIONS AND RECOMMENDATIONS |
Annex I — List of the projects and programmes examined
Annex II — Effectiveness and prospects of sustainability of projects and programmes examined
The Commission's replies
GLOSSARYACP African, Caribbean and Pacific Countries.Financing agreement Document signed between the Commission and the partner country, describing a particular project or programme to be funded. It represents the formal commitment of the European Union and the partner country to finance the measures described.Financing decision Decision taken by the Commission, on the basis of a financing proposal, to commit funds to a particular project or programme.Financing proposal Draft document prepared by the Commission services, submitted to the relevant Financing Committee of Member States' representatives for an opinion, and having received a favourable opinion, submitted to the Commission for decision. The document describes the background, nature, scope and objectives, implementation measures and the amount of funding foreseen for a particular project or programme.MDGs Millennium Development Goals consisting of eight goals to be achieved by 2015 — which range from halving extreme poverty to halting the spread of HIV/AIDS and providing universal primary education — were agreed upon by all countries and the world's leading development institutions at the UN Millennium Summit of September 2000.OECD/DAC Development Assistance Committee of the Organisation for Economic Cooperation and Development.Paris Declaration on Aid Effectiveness Statement adopted at a High Level Forum in Paris, 28 February — 2 March 2005 by a large number of countries and organisations, containing a number of commitments concerning ownership, harmonisation, alignment, results and mutual accountability, all with the aim of increasing aid effectiveness.
EXECUTIVE SUMMARY
I. |
Technical assistance is a resource used in many development projects and programmes of different types and can be defined as ‘experts contracted for the transfer of know-how and skills and the creation and strengthening of institutions’. In recent years, it has been increasingly used to support capacity development of central and local government services in beneficiary countries, in areas such as public finance management, health and education, among others. This change of focus towards the functioning of government services has been the result of a growing awareness among donors, including the European Union, that a lack of institutional capacity is a fundamental obstacle to sustainable development (see paragraphs 1 to 6). |
II. |
The purpose of the Court's audit was to determine the extent to which the Commission ensures that technical assistance contributes to institutional capacity development in beneficiary countries. This was done by assessing the Commission's approach towards capacity development and the use of technical assistance and its management of the design and implementation of technical assistance activities within capacity development projects (see paragraphs 7 to 8). |
III. |
Although institutional capacity building has been recognised in the European Community's development policy as a key element in the fight against poverty, the Commission has not included in its Country Strategy Papers a comprehensive assessment of institutional capacity weaknesses nor identified priority needs for capacity development within beneficiary governments' administrations and institutions. Neither does the Commission have guidelines on the role technical assistance should play in this context (see paragraphs 9 to 12). |
IV. |
Project choice is mostly relevant but design is often inadequate due to over-ambitious objectives, tight timeframes and insufficient assessment of the existing weak institutional environment and level of local ownership of the projects (see paragraphs 13 to 17). |
V. |
Project implementation is often hindered by a number of problems, including a long preparation and start-up phase, which makes the project design outdated by the time implementation starts and significantly reduces the time available for implementation. Recruitment of technical assistance often adds to the delays as the experts selected are frequently not available once contracts have been awarded. Current procurement procedures for contracting technical assistance are not conducive to the participation of public institutions, while in certain areas better qualified experts may be found within such organisations. Donors still provide technical assistance support mainly through separate, uncoordinated projects, while implementation arrangements do not encourage local ownership (see paragraphs 18 to 34). |
VI. |
Project reporting, monitoring and evaluation are in most cases adequate. However, there is no management information system in place to assess and record the performance of technical assistance companies or experts, and provide information when decisions on the procurement of technical assistance are taken (see paragraphs 35 to 39). |
VII. |
Project results vary considerably but the Court concludes that, in the light of the often difficult environment in which they take place, project results and the contribution made by the technical assistance used can be considered as reasonable. The quality of the technical assistance provided was generally found to be satisfactory, but other important factors have reduced the effectiveness of the support provided. Although the Commission does not have control over some of these factors, it should nevertheless better anticipate them (see paragraphs 40 to 43). |
VIII. |
On the basis of these observations, the Court makes detailed recommendations which could help the Commission to establish a more focussed and structured approach to the use of technical assistance, in order to provide more effective support for capacity development (see paragraphs 44 to 48). |
INTRODUCTION
1. |
Development cooperation is sub-divided into two types of cooperation or assistance: financial assistance and technical assistance, or in other words ‘money’ and ‘know-how’ or ‘ideas’. This report covers the Court's audit of technical assistance, which the Commission defines as ‘Specialists, consultants, trainers, advisers etc. contracted for the transfer of know-how and skills and the creation and strengthening of institutions’ (1). |
2. |
Technical assistance is a resource used in many development projects and programmes of different types, and its importance or effectiveness is judged through its contribution to the outputs and results of the related projects and programmes. Specific areas or tasks in the context of development cooperation, in which technical assistance has played or plays a role include:
|
3. |
The change of focus towards the functioning of central and local government services has been the result of a growing awareness among donors, including the European Union, that a lack of institutional capacity is a fundamental obstacle to sustainable development. Capacity development has therefore become one of the focal areas in the European Community's Development Policy of 2000 as well as in the new policy formulated in 2005 (2). |
4. |
The importance of capacity development has also been stressed in a recent OECD/DAC document (3), which states: ‘Adequate country capacity is one of the critical missing factors in current efforts to meet the Millennium Development Goals. Development efforts in many of the poorest countries will fail, even if they are supported with substantially increased funding, if the development of sustainable capacity is not given greater and more careful attention’ (4). |
5. |
Further evidence of the importance of capacity development can be found in the Paris Declaration on Aid Effectiveness of March 2005 (5), which states that partner countries' capacity to plan, manage, implement and account for results of policies and programmes is critical for achieving development objectives. It emphasises the need for donors to base their support on partner countries' national development strategies, institutions and procedures as much as possible and to provide support to address existing weaknesses in partner countries' institutional capacities. Although capacity development can be supported through a variety of other measures — such as training, scholarships, computer equipment and software, documentation and even physical infrastructure — technical assistance is the main instrument used in this context. |
6. |
The Commission's accounting and management information systems did not allow the Court to obtain precise and reliable information on the amounts spent on technical assistance, either by the European Development Funds or through the General Budget. The Court has estimated the total value of commitments for technical assistance contracts, which were signed during the period 2000-2005 and were still open at the beginning of 2006, at about 2 200 million euro (6). The share of technical assistance absorbed by capacity development activities could not be estimated. |
AUDIT SCOPE AND APPROACH
7. |
The purpose of the Court's audit was to determine the extent to which the Commission ensures that Commission-financed (7) technical assistance contributes to institutional capacity development in beneficiary countries. More specifically, the audit set out to answer the following questions:
|
8. |
The Court audited the Commission's approach and activities, at policy and project level, in relation to 12 beneficiary countries, selected from four different geographical areas receiving development assistance: Africa, Asia, Latin America and the Mediterranean region (8). Five of these countries were visited on the spot, while the other seven countries were covered through a documentary review and questionnaires sent to the Delegations and the beneficiary authorities of the respective countries. From an inventory of capacity development projects, 32 projects were selected which involved 33 technical assistance contracts being implemented during the period 2000-2006. The total amount committed to the projects audited was 400 million euro, with some 144 million euro devoted to technical assistance spending. The complete list of the 32 projects and programmes examined is included in Annex I. |
OBSERVATIONS
Does the Commission have a sound strategy for institutional capacity development, including the use of technical assistance?
9. |
The Commission translates the principles of the overall European Community's development policy into specific country strategies, usually covering five-year periods, by preparing Country Strategy Papers and more detailed National Indicative Programmes for all individual beneficiary countries. For the regions covered in the audit, the Court examined, on the basis of these documents (9), whether the Commission makes a systematic assessment of recipient countries' institutional capacity strengths and weaknesses, and whether these countries' overall capacity development needs are identified and prioritised, and translated into technical assistance needs. |
Inadequate institutional analysis in Country Strategy Papers
10. |
The examination of the Commission's relevant policy documents, for the countries covered in the audit, as well as the individual countries' own national development plans and/or so-called Poverty Reduction Strategy Papers, revealed a growing recognition of the importance of capacity development. However, the Commission did not present a comprehensive assessment at country level of institutional weaknesses, nor identify priority needs for capacity development within beneficiary governments' administrations and institutions. As a result, the selection of focal sectors for EC support was made without such information being available, with the risk that potentially important sectors or programmes in need of donor support may not have been identified. |
11. |
In 2005, the Commission prepared an internal Concept Paper on Institutional Assessment and Capacity Development, describing the meaning and importance of capacity development. The paper provides guidance on how to assess institutional capacity and support capacity development processes. However, these guidelines are not applicable to the Country Strategy Papers. At the lower level of sector plans greater attention is paid to capacity development issues — albeit in very general terms. These plans are prepared after the focal sectors to be supported in a specific beneficiary country have been decided. |
No guidance on technical assistance
12. |
Although technical assistance is a long-established and frequently used instrument in Commission-financed capacity development projects and programmes, the Commission has not defined the role of technical assistance and its contribution to capacity development. Neither are there specific guidelines for use by the Delegations when preparing projects, describing when and how to use the instrument in relation to capacity development. In addition, the ‘Twinning’ instrument, which has been successfully used to provide technical assistance to pre-accession countries (and more recently also to ‘Neighbourhood Policy’ countries), has so far not been available to the ACP-, Asian, or Latin-American countries. While the concept paper mentioned in paragraph 11 contains some considerations on technical assistance issues, it provides no guidance on the use of the various kinds of technical assistance in different situations, sectors or countries. |
Are technical assistance activities within capacity development projects well designed?
13. |
For the 32 projects audited, the Court assessed to what extent the choice of these projects was consistent with the Commission's, and the partner countries', national strategies, whether the project designs set clear objectives and related performance indicators and adequately justified the proposed type and amount of technical assistance. The Court further examined whether the beneficiary authorities had actively participated in project preparation, and provided clear indication of ownership of the project. |
Project choice relevant but design often inadequate
14. |
All the projects audited were consistent with the Commission's own Country Strategy Papers and with the sector priorities established therein, and they were generally in line with the overall development priorities of the beneficiary countries concerned. |
15. |
About three-quarters of the projects examined showed one or more of the following design weaknesses:
These factors, in turn, impede the proper formulation and implementation of the technical assistance tasks. |
16. |
Whereas all projects examined included technical assistance as a major input, in about half of the cases the project documents proposed certain types (international or local) and amounts (long-term or short-term assignments) of technical assistance, without giving clear reasons for the proposed combination of technical assistance resources. Furthermore, there was also a lack of clarity regarding the expected role of the experts because the advisory tasks relating to the project's capacity development objectives usually had to be combined with a ‘gap-filling’ role to support the institution concerned in its regular tasks. Box 1Example of project design weaknessesSupport Programme to the Health Sector — AngolaThe project objectives included strengthening institutional capacities at the central and provincial levels, improving basic health services, improving the availability of essential medicines and establishing a system for the prevention of HIV. Although the evaluation of a previous project had mentioned the difficulty found in ensuring an active role by the Ministry of Health, the feasibility study did not carry out an assessment of the local capacities to ensure the effective implementation of this ambitious project. Only when the project had started, was it noticed that some of the Angolan partners did not have adequate capacity to implement the project. The inception report prepared by the experts concluded that the project was too ambitious, but no changes were made until 18 months before the end of the project, when the objectives and activities were significantly reduced. |
Local ownership essential but not systematically addressed by the Commission
17. |
Ownership of the planned institutional development, on the part of the beneficiary institution and the people working for it is a key factor, determining the environment in which the technical assistance can operate and thereby contributing to the successful implementation of the capacity development plans. Being conscious of this, the Commission generally involved local beneficiary institutions in the design phase of its projects. However, it did not sufficiently assess the risk that a beneficiary government might not fulfil its commitments. The fact that a beneficiary government signs a financing agreement with a donor does not automatically imply that the assumptions, formulated in most project agreements about the government's commitment to reform, will be fulfilled. Nor does it mean that the project will be supported by all the people within the institution, or affected in some way by the project, as demonstrated by a lack of interest displayed in several cases by staff of the institutions receiving aid. Box 2Example of inadequate local ownershipHealth project — BeninThe project objective was to improve the quality and accessibility of health care services. However, the experts had to deal with many problems, such as the malfunctioning of the human resources management in the health sector, which was characterised by frequent transfers and arbitrary appointments, negatively influencing the motivation of staff. The weak involvement of the Ministry in the project caused many delays, e.g. it refused to second staff to act as local counterparts to the project management unit. Other staff supposed to act as contact points between the different directorates of the Ministry and the project lacked motivation because they considered that they should be paid for this activity which they regarded as ‘extra’ work. The Commission rightly opposes such payments. |
Are technical assistance activities implemented efficiently?
18. |
In order to assess whether the projects were implemented efficiently, the Court examined whether the project started on time and was implemented as planned, whether the procurement and mobilisation of technical assistance proceeded without undue delays, whether the level of coordination with other related projects or activities was adequate and whether the project implementation arrangements encourage ownership by the beneficiary institutions. |
Long preparation phase often leads to outdated project design
19. |
The preparation phase of the projects examined generally took between one and two years. It involved a number of required steps, including one or more preparatory consultants' missions (for identification and formulation), two rounds of advice and comments by the Quality Support Group (10) in the Commission's Headquarters, and submission of the financing proposal to the relevant Member States' advisory committee. |
20. |
After the proposal has been approved by the Commission, a procurement procedure follows (see paragraphs 22 to 26), in order to contract the required experts. By the time they arrive on the spot to start the project — which is in most cases at least two to three years after the design phase started — local conditions have often changed: there have been new political or other developments (elections, new appointments, new national policies), or other donors have started projects. As a result, the experts often find that the original project assumptions are no longer valid or some of the activities planned are no longer relevant. This means that in such cases the experts first have to review the beneficiary's current capacity development needs and revise the project design accordingly before they can start with the regular task of preparing the project's detailed work plan. |
21. |
However, such revisions rarely occur at the start of a project, because of the administrative work and time involved in getting approval to change a financing agreement. The Commission usually starts the project as designed and waits for the Mid-Term Review (normally carried out after some two years of implementation), before a decision is made about changes. By then, the time remaining to implement any necessary changes is very limited and overall effectiveness is compromised. Box 3Example of a long preparation period causing design to be outdatedEuropean Technical Assistance Programme for Vietnam (ETV2)The project was prepared with the help of two separate missions, in October 2000 and July 2002. The financing agreement was signed in April 2003, but for various reasons the technical assistance contract was not signed until August 2005. The project started in January 2006, mainly as a result of the original team leader no longer being available and having to be replaced. By that time 3,5 years had passed since the project was formulated and, given the rapid pace of change in Vietnam, many activities which had been defined were no longer relevant. A major task for the experts, therefore, during the inception phase was to redesign the project, with the result that less than half of the planned time was available for the implementation of the project activities. |
Problems during procurement and project start-up reduce time available for implementation
22. |
Technical assistance required for the implementation of projects is normally procured through an international restricted tender procedure (11). This procedure requires some six to eight months to complete. As the procurement procedure generally starts only after the financing agreement has been signed, it takes at least ten months before the experts are in place (eight months for procurement and two months for mobilisation of the experts). This period is taken into account when the formal implementation period of the project is determined in the financing agreement. |
23. |
However, the Court's audit of the 32 projects has shown that many problems have arisen during tender procedures, or were linked to the start of projects, which delayed the launching of project activities and reduced the available implementation period. Typical problems include:
The overall result was that the actual implementation period for practically all the projects examined had become, or was likely to become, too short to have all the planned activities implemented and the expected results achieved. |
24. |
Apart from the overall time limit set for project implementation, there is also a contracting deadline to respect (the so-called D + 3 rule), introduced in the 2003 Financial Regulations (12). Therefore, all the delays arising after the date of the Financing Decision, e.g. during the tender for the main technical assistance contract or during the start-up phase, effectively limit the period during which further implementation-related contracts for works, supplies or other services can be concluded. Several projects designed since this rule came into force have already been, or are likely to be, affected. As from 1 May 2007 a modification to the Financial Regulation applicable to the general budget (13) has come into effect, which eases the constraint of the D + 3 rule to some extent, with the 3 year period starting from the date of the financing agreement. However, any delay occurring after this date will still have the same effect of reducing the implementation period as well as the period available for contracting. |
25. |
It is technically possible to start the procurement process earlier — i.e. before the financing agreement is signed, or even before the financing decision is taken — in which case the procurement notice must be published with a ‘suspension clause’ (14). This would help to reduce the time period which normally passes between signing the financing agreement and the start of project activities. The use of such a clause is actually provided for, at least for the ACP countries, in one of the Annexes to the ACP-EU Partnership Agreement (the Cotonou-Agreement of 23 June 2000), ‘in order to ensure early project start-up’ (15). |
26. |
According to the Commission's internal guidelines on procurement this possibility should only be used in exceptional cases (without indicating what these could be). The Commission considers that there is a risk that tenders may have to be cancelled and that, if this occurs too frequently, the interest of potential service providers in future Commission-financed projects may be reduced. The suspension clause is therefore rarely used. However, the risk that a project is completely cancelled after the preparation has been completed and the financing decision has been taken is, in practice, limited. The Commission should clarify in its guidelines under what circumstances and conditions the suspension clause may be used. Box 4Example of reduced implementation period due to delaysDevelopment of Human Resources for Health — UgandaThe technical assistance contract was signed in September 2003, some 18 months after the financing agreement was signed, and the first experts arrived 2 months after that. A combination of factors had caused the tender procedure to have taken so long, including the transfer of the health training schools from the Ministry of Health to the Ministry of Education, and internal changes at the time in the Delegation (staff leaving, devolution, transfer of portfolios). Although the financing agreement had an implementation period of five years, and the technical assistance contract a performance period of four years, the 20 months' delay at the start had left only three years and three months for implementation. |
Experts often replaced after award of contract
27. |
In one-third of the contracts examined, almost immediately after they had won the contract for supplying technical assistants, the consultancy companies informed the Commission that key experts were no longer available. In some cases, this was due to the late completion of the tender evaluation and award of the technical assistance contract, which meant that experts were no longer committed as to their availability (16). In other cases, the nonavailability occurred within the validity period. The most common reason given was health problems. There is a risk that companies deliberately propose experts with good CVs in order to win the contract, while knowing that the expert will not be available to take up the assignment. In all cases, finding replacements delays the start up of the project. Box 5Example of replacement of expert after award of contractFollowing an efficient tender procedure, a consortium of several organisations won the technical assistance contract, principally on the strength of its proposed team leader. However, this team leader never took up his duties as he was withdrawn shortly after the contract was signed because it was discovered that he was susceptible to a disease which occurs in the country in question. This is something which should have been known earlier. His replacement arrived at the project almost three months later. |
Inappropriate selection criteria for technical assistance
28. |
The regular withdrawal of key experts raises the question as to whether the criteria used for evaluation of proposals for technical assistance are actually appropriate (17). The most important single criterion used in the technical evaluation of tenders for service contracts is the evaluation of the curriculum vitae of the long term experts proposed, in particular that of the team leader. Only in some cases are interviews with experts organised. Other potential indicators for technical quality are not taken into account, such as the overall capacity of the company, and/or the Commission's actual experience of the company or the proposed experts in previous Commission-financed contracts. |
Limited choice between procurement procedures
29. |
When it comes to the recruitment of experts for the implementation of capacity development projects, there is only a limited choice between different sources from which to recruit experts. There are basically two ways to recruit experts, i.e. through the Framework Contract system (18), or through a restricted international tender for services (see paragraph 22). |
30. |
In both cases the contract under which experts are to be hired is signed with a consulting firm. In certain areas of expertise such as public finance management or public administration, better qualified experts may be found within other types of organisation, such as the national administrations of Member States, e.g. in the case of public finance management, within national Treasuries, Ministries of Finance or Central Banks, similar to what happens in ‘Twinning’ projects in pre-accession countries. In cases where very specific local expertise is required, neither of the two procedures is in practice conducive to tapping into the local or regional market for technical expertise. |
Inefficient use of technical assistance
31. |
International experts were not always used efficiently, because of the different roles assigned to them. Although they are principally selected on the basis of their technical expertise, the Commission also expects experts to play a role in project management and financial control. Under the current Financial Regulation the international technical assistance team leader is no longer the Co-Director for the project with joint signatory powers, as was the practice under the previous Financial Regulation. However, in practice, the team leader not only verifies but also prepares the project work plans and budgets, the reports, tender dossiers, draft contracts and payment proposals from the project account. In some cases these administrative duties were taking up a disproportionate amount of time, thus reducing the time the expert(s) can spend on technical capacity development activities. It also leads to a certain ambiguity concerning the experts' accountability, i.e. to the beneficiary institution or the Delegation, which complicates their tasks and can compromise the climate of confidence between the expert(s) and the local counterparts. |
Donor coordination varies between countries
32. |
The level of coordination of capacity development programmes varies considerably between the countries concerned. This ranges from limited exchanges of information to more elaborate systems of sectoral and/or thematic working groups which bring together all active partners and display a high level of coordination, with the Commission usually being one of the most active donors in this respect. Where a programme of general budget aid or a sector policy support programme had been established, coordination was better organised and more intensive. However, even then the technical assistance was often still provided through separate projects, whereby each donor applied its own procedures. |
Implementation arrangements not yet favorable to local ownership
33. |
Project Implementation Units (or Project Support Units) are still set up in most countries as temporary structures, for the duration of a project and in parallel to the existing national administration, although in several cases they are already more integrated within the regular institutional structure of the beneficiary administration. In none of the countries concerned was the procurement of, and payment for, Commission-financed technical assistance being managed through national systems and procedures, as the Commission considered these to be unreliable and lacking transparency. However, several of these countries (i.e. Benin, Chad, Morocco, Sierra Leone, Uganda and Vietnam) are recipients of Commission-financed budget aid, whereby expenditure of the funds is, by definition, subject to the national financial procedures. This illustrates an incoherence in the Commission's approach regarding the use of the beneficiary countries' public financial management and procurement systems under its different aid instruments. |
34. |
In the context of the Paris Declaration (see paragraph 5) donors have undertaken a number of commitments to align their support with partner countries' development strategies, institutions and procedures. Several of these commitments are directly related to the issue of capacity development. In this context the European Union formulated in three areas even stricter targets for itself, i.e. to provide all capacity-building assistance through coordinated programmes with an increasing use of multi-donor arrangements; to channel 50 % of government to government assistance through country systems; and to avoid the establishment of any new Project Implementation Units. Given the design of most current capacity development projects, the Commission's task to meet its commitments under the Paris Declaration is a challenging one. |
Are technical assistance activities and performance adequately monitored and evaluated?
35. |
The quality and timeliness of reporting by the experts was generally satisfactory and projects are in most cases monitored actively by the Delegations' staff. In addition, there are two main systems in place for the external monitoring and evaluation of projects:
|
36. |
Both systems are used regularly. In some cases projects had been visited by the ROM consultants a short time after the start of activities, so that only the project design could be assessed and not the efficiency or effectiveness of implementation or the results. |
37. |
The ROM system represents good practice, particularly as it involves regular visits to projects during their implementation. However, the Court found in a number of cases that the scores given to projects were unrealistic or inappropriate or that the scores given by consecutive ROM visits to the same project were not consistent. This may be influenced by the fact that these visits are necessarily very brief (in view of the large number of projects) and because of regular changes of monitoring staff. |
38. |
Mid-Term Reviews are carried out for most projects and are generally seen as a very useful instrument providing a balanced, outside view on a project. Their recommendations are often used by the Delegations when deciding on changing, refocusing or extending projects. However, with the current trend of reducing implementation periods, it becomes more difficult to schedule a Mid-Term Review in such a way that a project has had sufficient time to progress, yet also leave sufficient time to implement and benefit from the recommendations. |
39. |
Systematic assessment of technical assistance performance, by companies and experts, does not take place. Any assessment carried out, either by a Delegation or by a local beneficiary or the local contracting authority (in the case of the EDF) is done on an ad-hoc and informal basis only. The Commission does not have any system to allow even this limited information to be shared. The lack of an adequate management information system in relation to technical assistance means that valuable experience from the past in working with certain firms or experts cannot be exploited when new decisions on the procurement of technical assistance are taken. |
Are technical assistance activities effective in terms of capacity development?
Technical assistance contribution satisfactory but success of projects not guaranteed
40. |
Overall, the actual or likely results for the 32 projects audited vary considerably (see Annex II). Ten projects were still at too early a stage of implementation to allow any reasonable anticipation of their likely results. As far as the remainder of the projects is concerned:
Box 6Example of a successful projectSupport to decentralisation in BeninThe project's objective was to establish at municipal level the administrative capacity to provide an improved level of public services to the population.An innovative approach was followed whereby a single international expert, responsible for leading and coordinating activities at central level, was combined with a large team of local experts, who had the specific local knowledge and provided support at the local level. This required an initial training of the local teams by the international expert in order to ensure a common approach and develop standardised tools. The project design included clear objectives and performance indicators, although they were not initially quantified. Mobilisation of the expert was within six months of signing the financing agreement. Upon the start of the project the expert refined the logical framework and established a database for monitoring and evaluation. Despite a difficult environment, due to the lack of clear political support for the decentralisation process, the project has largely achieved its objectives. The Commission, taking a long-term view towards development in this area, has prepared a second phase for the project, to support the implementation of the municipal development plans and to increase the capacity of the supervisory institutions. |
41. |
As technical assistance is the main input used in capacity development projects, its quality obviously has a significant influence on the project results. Performance by experts was generally considered satisfactory by the beneficiary institutions and the Delegations. Although in about half of the projects audited there have been cases of replacement of experts, this was usually due to personality differences but sometimes to underperformance. There have been only a few cases where a whole team of experts did not perform well and had to be replaced. However, good quality technical assistance does not necessarily ensure the success of a project, as other factors have a decisive influence on the outcome. |
42. |
For the large majority of projects, an insufficient implementation period has been a major contributory factor to projects not being able to produce their results within the time allocated by the financing agreement. The duration of many of these projects has been, or will be, extended, which should lead to further results being obtained. Extensions of the time period alone are not sufficient to guarantee a successful completion of projects. Other important factors influencing the effectiveness of the projects are the institutional environment in the beneficiary countries and the degree of ownership displayed by the beneficiary institutions during the preparation and implementation of the projects. For the countries visited by the Court, these factors have been generally positive in Vietnam and Uganda, and to a lesser extent in Algeria and Benin, but this was not the case in Angola. Such variations in local institutional capacity and ownership go a long way towards explaining the differences in the results obtained or expected from the projects audited by the Court in the respective countries. Some of these factors are outside the direct control of the Commission, which nevertheless should better anticipate them. Box 7Example of a project with partial results and uncertain sustainabilityTraining of primary school teachers — AngolaPoor project preparation and delays led to half of the planned activities being dropped. As a result, the project did not achieve its full objectives, as only 4 training centres are operational, against a target of 16, and 11 000 teachers were trained compared with an original target of 14 250. The government of Angola has not respected its commitments to finance these provincial training centres. Without classrooms, water or electricity, their infrastructure was totally inadequate and courses were taking place in the primary schools, obliging the children to have their courses in the playground. Teachers were generally happy with the training provided, but they had not yet received the formal certification by the Ministry of Education and therefore did not yet have the related professional upgrading or salary increases. A statistical data system was installed but not working. Other equipment provided by the project was not working due to lack of maintenance and spare parts. Box 8Example of a project with unsatisfactory resultsHealth Sector Support Programme — Sierra LeoneProject design took place under difficult circumstances in the late nineties, when the security situation was still very unstable. The project objectives were overly ambitious, in view of this difficult situation and the complexity of the decentralisation process. Delays in recruitment and inadequate performance of the experts, as well as the lack of ownership by the Ministry of Health, which did not provide the required local counterpart staff, have seriously affected the activities of the project. As a result, the project has fallen far short of its objectives. At the end of 2006, the Commission's services were preparing a rider to the financing agreement to decommit 13 million euro (from the original contribution of 28 million euro) and extend the project by two and a half years with a view to try to complete parts of the two more successful components, i.e. the primary health care at decentralised level and the pharmaceutical component. |
Sustainability uncertain
43. |
The prospects for the project results to be sustainable, i.e. that any improvement in capacity will be permanent, are in many cases uncertain (see Annex II). The inadequate implementation periods of many projects has led to a great deal of pressure to get project activities completed as quickly as possible, while not creating the proper conditions for a gradual withdrawal from the project of the external assistance. Sustainability will depend mostly on the commitment and the capacity of the beneficiary governments and institutions to continue to support development and implementation of reforms in the functioning of central government services, and more generally of public administration, after the Commission's support has come to an end. Apart from the political will required for such continued support, there are usually also financial implications, in terms of increased costs to the national or local government budget. In many countries audited, this creates significant doubts about the continuation of many reform and capacity development processes without on-going technical and/or financial assistance. |
CONCLUSIONS AND RECOMMENDATIONS
Does the Commission have a sound strategy for institutional capacity development, including the possible use of technical assistance?
44. |
Although institutional capacity building has been recognised in the European Community's development policy as a key element in the fight against poverty, the Commission has not included in its Country Strategy Papers an overall assessment and prioritisation of existing institutional weaknesses in individual countries, nor is such information used for the selection of focal sectors for EC support. Neither does the Commission have guidelines on the role technical assistance should play in this context (paragraphs 9 to 12). Recommendation 1In its Country Strategy Papers, the Commission should make a comprehensive and structured analysis of existing institutional capacity weaknesses and of capacity development needs.Recommendation 2The Commission should develop guidelines on technical assistance defining its role in the area of capacity development and providing a sound approach and tools to consider when and how to use it. |
Are technical assistance activities within capacity development projects well designed?
45. |
At the project level, technical assistance-related capacity development projects selected for support are mostly consistent with the, usually very widely formulated, overall national development strategies of the countries concerned, but they often lack a detailed justification of the specific technical assistance needs. Project objectives are frequently over-ambitious, given the tight timeframes set, and do not take sufficiently into account the weak institutional environment in which the projects take place, the level of local ownership of the project and the commitment to the proposed institutional changes, thereby impeding the proper formulation and implementation of the technical assistance tasks (paragraphs 13 to 17). Recommendation 3Design of capacity development projects should be improved, by facilitating effective ownership and leadership of the national part of the process, by better defining specific capacity development objectives and related technical assistance requirements, by avoiding overly complex implementation structures, by being more realistic in terms of objectives to be achieved and by planning longer implementation periods. |
Are technical assistance activities within capacity development projects implemented efficiently?
46. |
For a number of reasons, implementation periods are in most cases too short to implement all project activities and achieve the project objectives. This, in many cases, leads to a necessary extension of the project period. The main problems affecting project implementation, thereby reducing the technical assistance effectiveness and efficiency, are the following:
Recommendation 4The procedures governing the project preparation and start-up phase, including the procurement of technical assistance, should be reviewed, in order to create more time for implementation, and more flexibility should be allowed during the inception phase to adjust the project design and/or the Terms of Reference for the technical assistance to changes in circumstances.Recommendation 5The evaluation criteria in technical assistance tenders should be reviewed, in order to better reflect the quality and previous experience of the experts and the consultancy company.Recommendation 6More options should be considered regarding procurement possibilities to allow the best possible choice of technical expertise, including expertise from public institutions and expertise available in the beneficiary country or the region.Recommendation 7In line with the Paris Declaration commitments, the Commission should increase its use of technical assistance through coordinated programmes and apply, where possible, implementation arrangements which encourage local ownership. |
Are technical assistance activities and performance effectively monitored and evaluated?
47. |
The Court found that the reporting, monitoring and evaluation of technical assistance activities were in most cases adequate. Project reporting, monitoring and evaluation are carried out regularly and used to identify and address problems and to adjust and refocus projects where necessary. However, there is no system in place to assess and record the performance of technical assistance companies or experts, which could be used for future reference when experts or technical assistance companies are being recruited for Commission-financed projects (paragraphs 35 to 39). Recommendation 8Technical assistance performance by companies and experts should be assessed systematically and a management information system for recording, reporting and consulting this performance should be developed. |
Do technical assistance activities produce satisfactory results in terms of capacity development?
48. |
In the light of the difficult political and institutional environment in which they take place, overall, the results of Commission-financed capacity development projects can be considered as reasonable. While the quality of the technical assistance provided was generally found to be satisfactory, other important factors have reduced the effectiveness of the support provided. Capacity development is a difficult task, the more so because, as many of the projects audited have shown, existing institutional capacity is often weak and local commitment to institutional changes not always strongly present. |
This report was adopted by the Court of Auditors in Luxembourg at its meeting of 17 October 2007.
For the Court of Auditors
Hubert WEBER
President
(1) In this report the term ‘expert’ will be used to denote the different providers.
(2) Communication from the Commission to the Council and the European Parliament: The European Community's Development Policy, 26.4.2000 COM(2000) 212 final.
Joint Statement by the Council and the representatives of the Governments of the Member States meeting within the Council, the European Parliament and the Commission: ‘The European Consensus on Development’, General Affairs and External Relations Council on 22 November 2005 (14820/05).
(3) DAC Network on Governance: ‘The challenge of capacity development: working towards good practice’, February 2006. DAC is the Development Assistance Committee of the Organisation for Economic Cooperation and Development (OECD).
(4) Millennium Development Goals or MDGs: see www.un.org/millenniumgoals and glossary.
(5) See www.aidharmonization.org and glossary.
(6) Divided almost equally between the EDFs (covering all ACP countries) and the general budget (covering the three main geographical programmes for the Middle East and southern Mediterranean, Latin America and Asia).
(7) Either through the General Budget of the European Union or through the European Development Funds.
(8) The 12 countries are: Algeria, Angola, Benin, Uganda, Vietnam (all visited), and Chad, Indonesia, Mexico, Morocco, Nigeria, Peru, Sierra Leone (documentary review).
(9) At the time of the audit the current Country Strategy Papers (CSPs) covered the period 2002-2007 (for the EDF-countries) or 2002-2006 (for the non-EDF countries). Where new draft CSPs for the period 2007/8-2013 were already available, these were also examined.
(10) Quality Support Groups are Commission inter-service groups which perform peer reviews of external assistance measures managed by EuropeAid during their identification and formulation.
(11) Practical Guide to contract procedures for EC external actions: for services contracts with a value of 200 000 euro or more an international restricted tender procedure is compulsory.
(12) Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 on the Financial Regulation applicable to the general budget of the European Communities (OJ L 248, 16.9.2002, p. 1), Article 166; and the Financial Regulation of 27 March 2003 applicable to the 9th EDF (OJ L 83, 1.4.2003, p. 1), Articles 53 and 54. This rule requires contracts for works, supplies or services to be concluded not later than 3 years after the date when the Commission takes the Financing Decision concerning a project.
(13) Council Regulation (EC, Euratom) No 1995/2006 of 13 December 2006 amending Regulation (EC, Euratom) No 1605/2002 on the Financial Regulation applicable to the general budget of the European Communities (OJ L 390, 30.12.2006, p. 1), Article 100. The Financial Regulation applicable to the 9th EDF has not yet been modified in this respect.
(14) This means that the award of the contract will be subject to the conclusion of a financing agreement and the provision of funding.
(15) ACP-EU Partnership Agreement, Annex IV — Implementation and Management Procedures, Article 19.
(16) Experts are required to sign a statement that they will be available during the validity period of the offer, which is usually 90 days.
(17) According to the Commission's guidelines, the technical aspects (i.e. the ‘quality’) of the services are given a ‘weight’ of 80 % in the overall assessment. The financial aspect is not only determined by the remaining 20 % weight in the combined assessment, but also by the fact that all offers should stay within the limit of the available budget.
(18) Because of frequent use of short-term service contracts throughout the project cycle, the Commission has established, through an international tender, so-called Framework Contracts (FWC). These are global contracts, which establish the essential terms (e.g. duration, subjects, prices, and conditions of performance) for the award of specific, individual contracts required during a given period.
ANNEX I
LIST OF THE PROJECTS AND PROGRAMMES EXAMINED
(euro) |
||
|
Project Amount |
TA Amount |
COUNTRIES VISITED |
||
Algeria |
||
Support Programme for the reform of telecommunications and postal and development services of the Information Society (PTT) |
17 000 000 |
13 750 742 |
Programme of modernisation and assistance in respect of administrative reforms (MARA) |
25 000 000 |
10 899 900 |
Support Programme for education reform (Education) |
17 000 000 |
8 997 400 |
Support for the reform of justice in Algeria (Justice) |
15 000 000 |
6 175 960 |
Angola |
||
Training of Primary School Teachers |
6 000 000 |
950 000 |
Support Programme for the Health Sector |
25 000 000 |
7 150 000 |
Support for the National Institute of Statistics |
1 900 000 |
475 000 |
Strengthening the Planning and Budget Process |
13 500 000 |
6 500 000 |
Benin |
||
Support for the Health Sector |
10 200 000 |
2 160 000 |
Support Programme for the starting-up of communes — PRODECOM |
8 850 000 |
4 681 900 |
Support for the Integrated Programme for strengthening legal and judiciary systems |
12 150 000 |
1 394 630 |
Uganda |
||
Support for National Authorising Officer's (NAO) Office |
1 725 000 |
1 725 000 |
Poverty alleviation budgetary support IV (TA component only) |
1 190 000 |
921 540 |
Development of Human Resources for Health |
17 000 000 |
2 874 844 |
Technical Assistance to the Road Agency Formation Unit |
1 999 500 |
1 746 788 |
Vietnam |
||
European Technical Assistance Programme for Vietnam (ETV2) |
11 034 800 |
5 850 560 |
Support for the Renovation of Education Management (SREM) |
12 000 000 |
2 979 620 |
Institutional Support Project (ISP) |
8 000 000 |
2 549 100 |
Poverty Reduction Support Credit 3 (PRSC3) (TA component only) |
4 500 000 |
4 100 000 |
COUNTRIES NOT VISITED |
||
Indonesia |
||
Trade Support Programme |
8 500 000 |
5 488 682 |
Mexico |
||
Strengthening and modernisation of justice administration in Mexico |
4 600 000 |
2 071 825 |
Facilitation of the EU-Mexico free trade agreement (PROTLCUEM) (EU contribution) |
8 000 000 |
4 249 580 |
Morocco |
||
Modernisation of jurisdictions (EU contribution) |
27 671 000 |
4 634 720 |
Support for implementing the Association Agreement (PAAA) — Stage I |
5 000 000 |
5 000 000 |
Nigeria |
||
Economic Management Capacity Project (EMCAP) |
12 000 000 |
939 300 |
Support for reforming institutions programme (SRIP) |
27 600 000 |
10 780 409 |
Peru |
||
Support for the vocational education and training system of Peru (APROLAB) |
6 000 000 |
1 200 000 |
Programme in support of the reform of the Peruvian system of justice (JUSPER) |
12 000 000 |
1 400 000 |
Sierra Leone |
||
Institutional Support for the Ministry of Finance |
4 500 000 |
1 262 016 |
Health sector support programme (HSSP) |
28 000 000 |
7 533 000 |
Chad |
||
Support for health policy |
42 084 965 |
9 757 541 |
Budgetary support programme for the reduction of poverty and growth (2003-2006) (TA component only) |
4 000 000 |
4 000 000 |
Total |
399 005 265 |
144 200 057 |
NB: For the budget support programmes, only the TA component is taken into account in the project amount. |
ANNEX II
EFFECTIVENESS AND PROSPECTS FOR SUSTAINABILITY OF PROJECTS AND PROGRAMMES EXAMINED
|
Effectiveness |
Prospects of sustainability |
||||||
Successful |
Partially successful |
Unsuccessful |
Too early |
Good |
Uncertain |
Doubtful |
Too early |
|
COUNTRIES VISITED |
||||||||
Algeria |
||||||||
Support Programme for the reform of telecommunications and postal and development services of the Information Society (PTT) |
|
× |
|
|
|
× |
|
|
Programme of modernisation and assistance in respect of administrative reforms (MARA) |
|
|
|
× |
|
|
|
× |
Support Programme for education reform (Education) |
|
|
|
× |
|
|
|
× |
Support for the reform of justice in Algeria (Justice) |
|
|
|
× |
|
|
|
× |
Angola |
||||||||
Training of Primary School Teachers |
|
× |
|
|
|
× |
|
|
Support Programme for the Health Sector |
|
× |
|
|
|
× |
|
|
Support for the National Institute of Statistics |
|
|
× |
|
|
|
|
× |
Strengthening the Planning and Budget Process |
|
|
|
× |
|
|
|
× |
Benin |
||||||||
Support for the Health Sector |
× |
|
|
|
|
× |
|
|
Support Programme for the starting-up of communes — PRODECOM |
× |
|
|
|
|
× |
|
|
Support for the Integrated Programme for strengthening legal and judiciary systems |
|
|
|
× |
|
|
× |
|
Uganda |
||||||||
Support for National Authorising Officer's (NAO) Office |
× |
|
|
|
|
× |
|
|
Poverty alleviation budgetary support IV (TA component only) |
× |
|
|
|
× |
|
|
|
Development of Human Resources for Health |
|
× |
|
|
|
× |
|
|
Technical Assistance to the Road Agency Formation Unit |
|
× |
|
|
|
× |
|
|
Vietnam |
||||||||
European Technical Assistance Programme for Vietnam (ETV2) |
|
|
|
× |
× |
|
|
|
Support for the Renovation of Education Management (SREM) |
|
|
|
× |
|
× |
|
|
Institutional Support Project (ISP) |
|
|
|
× |
|
|
|
× |
Poverty Reduction Support Credit 3 (PRSC3) (TA component only) |
|
|
× |
|
|
× |
|
|
COUNTRIES NOT VISITED |
||||||||
Indonesia |
||||||||
Trade Support Programme |
× |
|
|
|
× |
|
|
|
Mexico |
||||||||
Strengthening and modernisation of justice administration in Mexico |
|
× |
|
|
|
× |
|
|
Facilitation of the EU-Mexico free trade agreement (PROTLCUEM) (EU contribution) |
|
|
|
× |
|
|
|
× |
Morocco |
||||||||
Modernisation of jurisdictions (EU contribution) |
|
|
× |
|
|
|
|
× |
Support for implementing the Association Agreement (PAAA) — Stage I |
× |
|
|
|
× |
|
|
|
Nigeria |
||||||||
Economic Management Capacity Project (EMCAP) |
|
× |
|
|
|
|
× |
|
Support for reforming institutions programme (SRIP) |
|
|
|
× |
|
|
|
× |
Peru |
||||||||
Support for the vocational education and training system of Peru (APROLAB) |
|
|
× |
|
|
|
|
× |
Programme in support of the reform of the Peruvian system of justice (JUSPER) |
|
|
× |
|
|
× |
|
|
Sierra Leone |
||||||||
Institutional Support for the Ministry of Finance |
× |
|
|
|
|
× |
|
|
Health sector support programme (HSSP) |
|
|
× |
|
|
|
× |
|
Chad |
||||||||
Support for health policy |
|
|
× |
|
|
× |
|
|
Budgetary support programme for the reduction of poverty and growth (2003-2006) (TA component only) |
|
× |
|
|
|
× |
|
|
Total |
7 |
8 |
7 |
10 |
4 |
15 |
3 |
10 |
THE COMMISSION'S REPLIES
EXECUTIVE SUMMARY
I. |
In the last 3-4 years the thinking on capacity development and technical assistance has substantially evolved (in line with the aid effectiveness debate), and in 2005 and 2006 a set of good practices in these areas have gradually been agreed upon in the donor community. This new thinking has been formally acknowledged by the Paris Declaration that includes various commitments in this area. Results from the first monitoring exercise on Paris indicators from the DAC (report of July 2007) show that reforming technical assistance is a complex and long-term process, challenging for all donors where partner governments also play a key role. |
III. |
The country analysis of all Country Strategy Papers (CSPs) gives sufficient information to prepare the response strategy and to choose the focal sectors, where institutional aspects are covered, especially where a sector approach is foreseen, or where capacity development is a major area of the Commission intervention. Capacity development and technical assistance issues are addressed in different guidelines. The use of technical assistance within institutional capacity building must be addressed in a flexible way, taking into account that good practices for both external and local technical assistance by governments vary a lot from one sector to another and from one country to another. |
IV. |
The Commission attaches a particular importance to the quality of project design and has a system to ensure quality of project design; at the identification and at the formulation level. The system was fully developed in 2005. Most of the projects examined in this report were prepared before that date. One of the main goals of the system is to tackle the problems during project preparation, including fixing objectives and assessing the existing institutional environment and level of local ownership of the projects. |
V. |
Regarding donor coordination, the Court's statement can be accepted for the projects that have been audited. However, the issue has been regularly addressed in the last few years and one of the main recent achievements is the adoption of the Paris Declaration in March 2005. The use of coordinated technical assistance also depends on other donor policies and practices and on partner government leadership. The Commission will continue to promote this type of cooperation while acknowledging that many actors intervene in these processes. |
VI. |
Technical assistance performance is monitored during project implementation and the Commission takes necessary measures, including the replacement of the experts, where such performance is not in line with contractual obligations. A ‘Contractor Assessment Form for EC-Funded Service contracts’ has to be completed by each Project Manager for every service contract and attached to the final report. On this basis, the Project Manager will rate the performances of the contractor and of the key experts. Regarding the award of new contracts, an analysis of the technical capacity of the candidate firms, including previous contract management experience, is conducted during the pre-selection procedure, where contract references are requested to applicants. The Commission, applying objective criteria, respects the principle of equal treatment of candidates. See also the Commission's reply to paragraph 28. |
VIII. |
In July 2007 the Commission launched a process to develop a strategy to meet aid effectiveness targets on Technical Cooperation and Project Implementation Units. Consultation with different relevant Commission services, EU Member States and partner countries is a key component of this strategy. This strategy in fact aims at the development of a structured approach, as suggested by the Court, based on a broad consultation. It addresses many of the issues highlighted by the Court's report and will consider its recommendations in order to ensure effective support to capacity development in line with EU targets in this area. |
INTRODUCTION
1. |
The Commission underlines the increasing importance of general and sector policy dialogue as a ‘third’ type of development cooperation, particularly with respect to the support to sector programmes. The new role of technical assistance in this context will need to be analysed and new technical assistance needs and requirements will need to be defined. |
5. |
The Commission underlines the importance it attaches to other dimensions of capacity building. |
6. |
The Common Relex Information System (CRIS) reporting follows the DAC sectoral codes but requires continuous improvements in line with evolving management needs. It is being considered whether information systems should be modified to make it possible to extract data on technical assistance components (where this is significant). |
OBSERVATIONS
10. |
In July 2007 the Commission launched a process to develop a strategy to meet aid effectiveness targets on Technical Cooperation and Project Implementation Units. Consultation with different relevant Commission services, EU Member States and partner countries is a key component of this strategy. The Commission considers that the analysis of Country Strategy Papers gives sufficient information to prepare the response strategy and to choose the focal sectors, where institutional aspects are covered, especially where a sector approach is foreseen, or where capacity development is a major area of the Commission intervention. The Commission's approach to deal with the growing recognition of the importance of capacity development is to mainstream institutional aspects in a holistic approach rather than to undertake a preliminary and systematic analysis of institutional aspects. Moreover, institutional aspects are further analysed during the programme design phase. In order to assess the eligibility to budget support for example, a detailed assessment is systematically made of the public finance management system. |
11. |
This Concept Paper was originally conceived to support institutional and capacity analysis at sector level as a tool to facilitate design and implementation of the Sector Wide Approach. This document was actually developed to complete the Guidelines on EC support to Sector Programmes issued in 2003. Its content can be easily adjusted to be applied for sub-sector/thematic analysis and at project level. However, this methodology (based on the identification of a sector ‘vantage point’) cannot be fully applied for ‘country-wide assessments’ that are very complex and delicate exercises, requiring ad hoc approaches. Building on this Concept Paper the Commission is developing further its methodology for institutional assessment at sector level. The new Guidelines on EC Support to Sector Programmes (July 2007) provide further guidance on this. Additional tools are being piloted. The Commission organises training on these issues also at country level. |
12. |
Capacity development and technical assistance issues are addressed in different guidelines, including the guidance on Sector and Budget Support (the recently issued Guidelines on Support to Sector Programmes — July 2007 and the Guide on General Budget Support — January 2007) and sector specific guidance. Preparing all-encompassing guidelines for the Delegations on the potential role of technical assistance in the specific and very sensitive field of institutional capacity development is a delicate issue that needs to be addressed in a flexible and context-sensitive way. The Commission has decided to prepare a participatory and comprehensive strategy in line with the Paris commitments and EU targets. Twinning projects cannot be launched before appropriate structures of recipient administrations are in place and a pipeline is prepared. Extension to other regions needs careful consideration, in order to ensure adequate implementation and response from EU partners. |
15. |
Second indent Several improvements have been introduced in the latest revision of the Financial Regulation and its implementing rules. Among these, the modification of the so-called ‘d + 3 rule’ should facilitate project implementation by linking the contracting time to the real duration of the project. Third indent Accountability is one of the pillars of the Paris Declaration on aid effectiveness. Strengthening measurable, results-oriented indicators is a priority for the new programming cycle. Fourth indent Better assessment of the institutional and capacity issues will be promoted through guidance and training. |
16. |
Technical assistance contracts are mainly used where a service provider is called on to play an advisory role, to manage or supervise a project, or to provide the experts specified in the contract. The distinction between international and local experts has however been abolished as of 1 February 2006. The new Guidelines on technical assistance will allow more clarity on technical assistance roles starting from the identification and formulation phases. |
Box 1
The health sector support programme continued to support sectors already covered in the previous project (7 ACP Angola 35), but still under very difficult circumstances (post-conflict environment, institutional weakness, decentralisation process). It is true that the start-up report noted the overambitious nature of the objectives set by the programme. The action plans and programme estimates were in fact drawn up on the basis of the recommendations of that initial report. Today the programme is active, useful and appreciated by central and provincial partners in the Ministry of Health.
17. |
Capacity development processes are inherently political where many national actors with possibly diverging interests and agendas intervene. The Commission recognizes the crucial role of local commitment for successful implementation of its support to capacity development and agrees that assessing commitment and ‘real demand’ deserves special attention. In this respect, methodological work is ongoing in order to develop an analytical tool addressing the various factors influencing real ownership and supporting realistic assessment of national commitment/interest. This tool and these approaches will need to be carefully piloted and progressively applied giving attention to the political dimension and sensitivity. At the same time, the Commission has the possibility to suspend, if needed, the implementation of a financing agreement when the beneficiary state breaches one of its obligations. |
19. |
The duration referred to by the Court is classical in co-operation activities, whatever the donor (World Bank, UN Agencies, bilateral co-operation agencies). It largely results from the project cycle. |
20. |
A checking and updating exercise is generally requested from the technical assistance during the inception phase as part of the terms of reference of their assignment. |
21. |
The original Logical Framework Matrix may sometimes not be complete, or may be out of date by the time the project starts, and therefore needs to be reviewed and, as appropriate, improved and/or updated during project implementation. Amendments to financing agreements and consultants' terms of reference are common practice. |
Box 3
In view of the delayed start-up, an amendment to the financing agreement is currently being processed by the Commission to give the project sufficient implementation time and to fully adjust the project to the current needs of Vietnam. The mid-term evaluation confirmed the approach taken by the Commission and supports the amendment to the financing agreement.
22. |
There are strict rules governing the way in which contracts are awarded. These help to ensure that suitably qualified contractors are chosen without bias and that the best value for money is obtained, with the full transparency appropriate to the use of public funds. The use of an international restricted tender procedure is foreseen by the implementing rules of the Financial Regulation (Budget) and the General Regulation (EDF) and can normally be completed in six or seven months. |
24. |
The new starting date of the so-called ‘d + 3 rule’ with the latest revision of the Financial Regulation already constitutes an improvement (see also the Commission's reply to paragraph 15). The d + 3 rule was introduced in 2003 by the legislator to find a balance between the necessity to control the outstanding amounts to be contracted (RAC) and the difficulties which can arise in the management of complex projects. It is also meant as an incentive for the beneficiary to implement financing decisions in a shorter period of time. |
25-26. |
The logic of the system of rules governing the EC general budget requires a financing decision by the Commission before the commitment (1) and call for tenders or proposals. Exceptional circumstances could arise where deviation from the normal decision-making process might be justified. As a rule, these require that the conditions which would justify a suspension clause are outside the Commission's control. Provisions concerning the European Development Fund do foresee the possibility for ACP States to use a suspension clause, in all duly substantiated cases and in agreement with the Commission, in order to ensure early project start-up. The Joint ACP/EU Council Decision No 2/2002 foresees the use of suspension clause tenders in exceptional and duly justified cases (Point 5 of its Annex). |
27. |
Measures are in place to deal with these cases which range from the possibility to cancel the award decision (if the withdrawal occurs before the signature of the contract) to the possibility to impose fines or exclude a company from future contracts. The Commission carefully analyses each case and ensures that, where appropriate, the replacement of the experts is of an equivalent quality. |
Box 5
The expert's replacement performed satisfactorily.
28. |
Interviews with team leaders are organised more systematically for projects entailing high level contacts. An analysis of the technical capacity of the candidate firms, as well as of their previous contract management experience, is conducted during the pre-selection procedure, which leads to the establishment of a short list. Contract references (up to 15) are requested from the applicants. The Commission verifies the satisfactory outcome of these when EC-financed contracts are mentioned. The drafting of the award criteria must be done carefully, in order that these are not mixed up with the selection criteria. Constant case-law of the Court of Justice recalls the importance to separate the evaluation of the selection criteria viewpoint and the evaluation of the award criteria. The ‘overall capacity of the company’ is usually assessed in the first phase of the procedure where it is established whether a company is capable or not of implementing the project (selection criteria) and may not be assessed a second time in the second phase of the procedure, where the quality of the offer is evaluated (evaluation/award criteria). The Commission respects the principle of equal treatment of candidates applying objective criteria. |
30. |
With respect to the participation of public institutions (from EU Member States) in technical assistance, there are important obstacles to the use of such technical assistance, in particular the need for prior professional experience in low income countries (present only with very few individuals) and the reluctance of institutions to delegate well qualified and well performing staff to work abroad. Competition is generally open to all natural and legal persons in eligible countries, including public bodies. Member States experts and local experts are not excluded. Publication of tender notices in local media is possible. New ways to find suitable experts can be explored, as long as they are consistent with the applicable rules (Financial Regulation, Implementing Rules, etc). |
31. |
Team leader management skills are essential to organise technical assistance inputs in a consistent and effective manner and to ensure adequate monitoring of technical assistance performance. As regards accountability, the contractor is always accountable to the contracting authority (if the contract is signed under decentralised management by the beneficiary country, the contractor is always accountable to the local institution concerned; if the contract has been concluded by the Commission, he will be accountable to the Delegation). It is not part of the standard Terms of Reference template that experts have to be given a role in project management and financial control. The Commission recognizes that technical assistance do play different roles (including administrative duties) and is screening this issue in the preparation of the future strategy on technical cooperation with the aim of ensuring more clarity and more accountability to partner governments. |
32. |
Donor coordination has always been a pivotal matter for the Commission and since 2005 important efforts have been deployed to promote structured coordinated actions, including technical assistance, in the area of capacity development. Coordination, alignment, ownership are now key elements of the aid effectiveness agenda and the Commission has made firm commitments to further enhance its operationalisation. The present context is conducive to coordinated technical assistance (2). |
33. |
The aid effectiveness agenda includes more integration of programme management in local administrations and phasing out PMUs. Since 2003, all EC financed Project Implementation Units are formally integrated in national administrations:
The Commission believes that there is therefore no incoherence in the Commission approach. |
34. |
The recently revised Financial Regulation for the EC general budget and the new instruments of EC external assistance (DCI, ENPI) as well as the implementing regulation for the 10th EDF increase the possibility for coordination with donors (co-financing rules in the new instruments and new management modes in Articles 53 to 57 of the Financial Regulation). The Commission agrees that meeting the Paris Declaration targets is a complex challenge. For this reason, its strategy on technical assistance and Project Implementing Units foresees a comprehensive desk work as well as a broad consultative phase. The recent survey from DAC on the monitoring of the Paris indicators on Project Implementing Units and Technical Cooperation (report of July 2007) shows that all donors share this same challenge. |
37. |
The Results Oriented Monitoring (ROM) has been fine-tuned in the course of the years and it is now largely recognised by all Commission services (Headquarters and Delegations) as an effective mechanism to have an impartial picture of the overall status of projects and support the regular monitoring mechanisms. |
39. |
See the Commission's reply to paragraph 28. |
CONCLUSIONS AND RECOMMENDATIONS
Recommendation 1
The Commission considers that the current approach already gives adequate consideration to relevant institutional aspects in the development of the programming documents. The Commission attaches particular attention to institutional aspects. These are taken into account through the entire cycle of operations starting from the programming phase.
The current format for a common framework for drafting the Country Strategy Papers (based on the Conclusions of the Council of 11 April 2006) clearly states that this document should include a joint analysis of the country situation, covering institutional aspects. On the basis of this assessment the choice of focal sector is made.
Recommendation 2
The Commission agrees to this recommendation. In fact, in order to meet the EU related targets, it has already been decided to develop a strategy to enhance the effectiveness of technical cooperation and deal with Project Implementation Unit issues. The overall purpose of this exercise, launched in early July 2007, is to improve the quality of technical cooperation with a focus on technical assistance personnel in line with capacity development issues and good practice in this area. The development of Guidelines on technical assistance will be part of this process benefiting from the broad consultation with delegations, Headquarter services, Member States and partner governments. Guidelines on technical assistance will be key in promoting a more structured and strategic approach. Nevertheless, given the complexity of this type of intervention, the Commission believes that these guidelines should support a flexible and context-sensitive approach (adaptable to country and sector situations), providing tools for a better understanding of the context specificities.
Recommendation 3
The Commission agrees to this recommendation. The user guidance on technical assistance will support better design of capacity development projects, including adequate consideration of technical assistance needs. Quality support groups will support implementation of this approach.
46. |
See the Commission's reply to paragraphs 19 to 21. Second indent See the Commission's reply to paragraphs 22 to 26. Third indent See the Commission's reply to paragraphs 27 to 28. Fourth indent See the Commission's reply to paragraphs 29 to 30. Fifth indent See the Commission's reply to paragraphs 32 to 34. |
Recommendation 4
The Commission has already introduced or is introducing a number of measures within the boundaries of the applicable legal framework, to follow this recommendation (see paragraph 24). The Commission is also screening current procedures against good practices and aid effectiveness principles in the area of technical assistance (desk phase of the EuropeAid Strategy on technical cooperation and Project Implementation Units).
Recommendation 5
The Commission finds that the existing evaluation criteria ensure that the principle of equal treatment of candidates is respected by applying objective criteria. See also the Commission's reply to paragraph 28.
Recommendation 6
As a general rule, the public procurement rules help to ensure that suitably qualified entities are chosen and that the best value for money is obtained, with the full transparency appropriate to the use of public funds. The participation of experts from public institutions should be considered in view of the competence and the experience necessary to carry out the technical assistance. See also the Commission's reply to paragraph 30.
Recommendation 7
The Commission is in the process of designing a strategy on the use of technical assistance and technical cooperation to increase aid effectiveness in line with the Paris Declaration. The strategy will further support coordinated actions with partner country and other donors, under partner country leadership.
Nevertheless, the use of coordinated technical assistance also depends on other donor policies and practices and on partner government leadership.
Recommendation 8
Technical assistance performance is monitored during project implementation and the Commission takes necessary measures, including the replacement of the experts, where such performance is not in line with contractual obligations. A ‘Contractor Assessment Form for EC-Funded Service contracts’ has to be completed by each Project Manager for every service contract and attached to the final report. On this basis, the Project Manager will rate the performances of the contractor and of the key experts.
Regarding the award of new contracts, an analysis of the technical capacity of the candidate firms, including previous contract management experience, is conducted during the pre-selection procedure, where contract references are requested to applicants. The Commission, applying objective criteria, respects the principle of equal treatment of candidates. See also the Commission's reply to paragraph 28.
(1) Article 75 of the Financial Regulation applicable to the general budget of the European Communities.
(2) Paris Declaration on Aid Effectiveness, March 2005; Aid Effectiveness Package, March 2006; Programming guidelines 10th EDF and common framework for joint programming; Council Conclusions on a Code of Conduct for Complementarity and Division of Labour in Development policy, May 2007.