52011SA0009

GLOSSARY

Broadband : high-speed data transmission in which the bandwidth is shared by more than one simultaneous signal.

DG INFSO : Directorate-General for the Information Society and Media

DG REGIO : Directorate-General for Regional Policy

EIF : European interoperability framework

ERDF : European Regional Development Fund

ICT : Information and Communication Technologies

IDA : interchange of data between administrations — programme aiming at promoting the development and operation of trans-European ICT networks for data interchange between Member State administrations and/or the Community institutions.

IDABC : interoperable delivery of European e-Government services to public administrations, businesses and citizens. Successor of the IDA instrument.

Interoperability : capability of different programs to exchange data via a common set of exchange formats, to read and write the same file formats, and to use the same protocols.

IS : Information Society

ISA : new programme for the period 2010–15 "Interoperability solutions for European public administrations".

NDP : national development plan

Open source software : broad general type of software license that makes source code available to the general public with relaxed or non-existent copyright restrictions.

Open standard : any communication, interconnection or interchange protocol, and any interoperable data format whose specifications are public and without any restriction in their access or implementation.

PLN : official ISO 4217 currency name for Polish złoty

PMM : Project Management Methodology is a process-based method for effective project management. The PMM provides a method for managing projects within a clearly defined framework. It describes procedures to coordinate resources and activities engaged in a project, giving indication on how to design and supervise the project, and how to react if the project has to be adjusted to respond to changes.

SF : Structural Funds

SMART : specific, measurable, achievable, realistic and timely

EXECUTIVE SUMMARY

I. Public administrations in the period covered by this report saw e-Government as a means of lessening the administrative burden of citizens, businesses and other administrations. It can also serve to improve the services they deliver. This is in keeping with the achievement of the Lisbon goals (competitiveness and innovation), the reduction of barriers to the internal market, the mobility of citizens across Europe, and regional development.

II. The Court’s audit focused on ERDF expenditure of the 2000–06 programme period in four Member States: France, Italy, Poland and Spain. The Court examined the relevant strategic and programming documents and visited 28 administrative projects developing e-Government. An online survey sent to a sample of project managers supplemented this work.

III. The Court examined whether these e-Government projects:

(a) have been selected according to assessed needs;

(b) have been well designed and delivered as planned;

(c) are useful and durable.

IV. The Court concluded that the e-Government projects supported by the ERDF have contributed to the development of electronic public services. However despite the fact that the majority of them are technically operational, due to insufficient focus on project results the benefits obtained are much lower than could have been expected.

V. More specifically the Court observed that:

(a) Despite the fact that e-Government programming at national level gradually improved during the programme period, weaknesses in early strategies meant that projects did not focus on addressing priority needs for e-Government development.

(b) Projects generally delivered their outputs. However, these were often delivered late or with a reduced scope due to poor design or lack of a sound implementation methodology.

(c) Most audited projects were technologically sound and developed IT applications providing electronic services to public bodies, businesses and citizens. The systems co-financed by the ERDF, in general were appropriately maintained and financially sustainable. However, in the absence of the measurement of actual project benefits, it is not possible to evaluate projects or accumulate knowledge for future programmes.

VI. Therefore the Court recommends that:

(a) Member States should develop strategies for e-Government, which are based upon identified needs, have clear objectives and assign responsibilities to the bodies accountable for the achievement of these objectives.

(b) Managing authorities should select e-Government projects for ERDF support on the basis of an assessment of likely project costs and benefits.

(c) Managing authorities in Member States should ensure that e-Government projects selected for ERDF funding focus not only on project outputs but also on the changes in processes or organisation necessary to fully benefit from the systems developed.

(d) Managing authorities should emphasise the practical application of best practice and strongly recommend the use of an appropriate project management methodology for e-Government projects in receipt of ERDF funding.

(e) The Commission should ensure that the EU principles and recommendations enabling trans-European interoperability, in particular the EIF principles, are considered by projects benefiting from the ERDF, in order to increase the European added value of a project and facilitate further EU-wide systems integration.

(f) When selecting e-Government projects, managing authorities should ensure that all significant costs, including maintenance have been sufficiently provided for in the cost-benefit analysis which should underpin the financing decision.

(g) The Commission should ensure that managing authorities monitor and evaluate project results and impacts in order to demonstrate the effective use of EU funding and provide important feedback to improve the design of future programmes.

INTRODUCTION

DEFINITION AND OBJECTIVES OF e-GOVERNMENT

1. Electronic Government (e-Government) refers to electronic transactions between government and citizens or businesses, or between different governmental administrations. e-Government requires the use of Information and Communication Technologies (ICT) combined with organisational change and new skills in order to deliver better quality public services [1].

2. Public administrations in the period covered by this report saw e-Government as a means of reducing administrative burden and waiting times, whilst improving cost effectiveness, and raising productivity. It was in line with the Lisbon strategy goals in competitiveness and innovation, aid regional development and reduce barriers in the internal market. It can also facilitate the mobility of citizens across Europe.

3. In order to avoid the emergence of so-called electronic barriers [2], the Commission recommended that European public administrations should respect a number of general principles, which were set out in the European interoperability framework [3] (EIF) published in November 2004.

LISBON STRATEGY, eEUROPE AND i2010

4. The Lisbon European Council held in 2000 set the ambitious objective of Europe becoming the most competitive and dynamic economy in the world and recognised the need for Europe to grasp the opportunities offered by the new economy and the Internet in particular.

5. The European Commission launched two successive eEurope Action Plans for the period 2000–05 aimed at stimulating services, applications and Internet content, covering both online public services and e-business. The action plans were also designed to address deficiencies in broadband infrastructure and security measures. These actions were to be reinforced by such activities as the dissemination of good practices and benchmarking [4] of ICT development in Member States.

6. From the outset [5], the Commission considered e-Government as important and the transformation of public services became a cornerstone of ICT policy. Following the eEurope2002 Action Plan, the Commission published a communication [6] expanding on the e-Government actions contained in the action plan and detailing the steps necessary for transforming public services with the use of ICT.

7. In 2005 eEurope Action Plans were replaced by a new strategic framework, i2010 — European Information Society 2010 — laying out broad policy orientations [7]. This renewed strategic framework put even more emphasis on e-Government. Indeed according to recent studies [8] there is a strong link between modern and efficient public administration and national competitiveness and innovation. A 2006 Commission communication [9], described previous achievements and set out new objectives for e-Government policy. The political commitment of Member States and other stakeholders is important for such actions and at a number of conferences [10] Member States demonstrated this by signing declarations to this effect. The first significant legal act [11], the "Services Directive" (see Box 1), published in 2006 and required to be in force in 2009, requires certain public services for business to be available electronically.

8. In parallel with its action plans, the Commission developed a number of initiatives [12] in specific fields with various objectives such as removing technological bottlenecks or facilitating exchange of best practice. Amongst more specialised programmes and initiatives, the IDABC [13] programme deserves mention as it facilitated cooperation between the Commission and Member States in working out common interoperable standards [14].

BOX 1

THE SERVICES DIRECTIVE

The directive requires Member States to simplify procedures and formalities that service providers need to comply with. In particular, it requires Member States to remove unjustified and disproportionate burdens and to substantially facilitate:

the establishment of a business, i.e. cases in which a natural or legal person wants to set up a permanent establishment in a Member State, and

the cross-border provision of services, i.e. cases in which a business wants to supply services across borders in another Member State, without setting up an establishment there.

Pursuant to the directive Member States are obliged to set up "points of single contact", through which service providers can obtain all relevant information and deal with all administrative formalities without the need to contact several authorities. The "points of single contact" have to be accessible at a distance and by electronic means.

EU FINANCIAL SUPPORT

9. Approximately 11 billion euro [15] of EU support was earmarked for the Information Society in the 2000–06 programme period. Amongst its aims were the development of new secure technologies, infrastructural capabilities and connectivity between regions through broadband. ERDF funding accounted for 6,7 billion euro (61 %) of this. No aggregate figures are available for e-Government actions.

THE PRINCIPLES OF ERDF MANAGEMENT IN THE 2000–06 PROGRAMME PERIOD

10. The ERDF management system reflects the principles of subsidiarity and partnership [16], with responsibilities and management shared between the European Commission and Member State governments. The ERDF programming and the implementation process of co-financed actions in force during the audited 2000–06 programme period is set out in Figure 1 in simplified form.

11. The multiannual programmes were drawn up by Member States following a process of dialogue with the relevant public authorities and the economic and social partners. Member States identified broad programme priorities, and, after negotiations, the Commission approved all programming documents. The broad description of measures was included in the operational programmes while the detailed measures and the choice of projects were made by Member States.

12. The programmes contain established and approved priorities to be implemented during the programme period. Each operational programme or single programming document has an associated programme complement which describes programme measures in detail. Measures may include specific actions for the development of e-Government.

FIGURE 1

SIMPLIFIED ERDF MANAGEMENT PROCESS FOR 2000–06

+++++ TIFF +++++

AUDIT SCOPE AND APPROACH

13. The main objective of the audit was to assess whether the audited e-Government administrative projects supported by the ERDF:

(a) have been selected according to assessed needs;

(b) have been well designed and delivered as planned;

(c) are useful and durable.

14. The Court’s audit focused on the 2000–06 programme period and four Member States: France, Italy, Poland and Spain who between them accounted for 3,0 billion euro (45 %) of expenditure from the ERDF on the Information Society.

15. The audit work included:

(a) a review of the Commission strategic framework on e-Government policy;

(b) an examination of national and relevant regional e-Government strategy documents for the 2000–06 period, in the countries visited, combined with an analysis of national development plans, relevant operational programmes and supporting programme complements;

(c) visits to national and regional authorities responsible for e-Government development and project selection in the visited Member States;

(d) an examination of 28 regional projects with total funding of 216,9 million euro and the ERDF contributing some 127,2 million euro of this. The average e-Government project visited had a value of 7,8 million euro with an average ERDF contribution of 4,5 million euro. Ten of the projects were of less than one million euro with the largest four projects ranging from 27 to 50 million euro. For the types of project examined (see the Table and Box 2);

(e) an online survey sent to 363 project managers of e-Government projects co-financed by the ERDF in the visited countries. Replies were received from 212 managers (58,4 %).

TABLE

TYPES OF e-GOVERNMENT PROJECTS VISITED ON THE SPOT

Type of project | Spain | France | Italy | Poland | Total |

Geographical Information System | 1 | 3 | 3 | 1 | 8 |

Workflow system | 3 | 1 | 2 | 4 | 10 |

Tax services | 1 | 0 | 0 | 1 | 2 |

Referential database | 0 | 0 | 1 | 1 | 2 |

Other | 2 | 3 | 1 | 0 | 6 |

Total | 7 | 7 | 7 | 7 | 28 |

BOX 2

EXAMPLES OF PROJECTS PROVIDING e-GOVERNMENT SERVICES

Geographical Information System (GIS) — Tuscany Region, Italy

The development of a GIS accessible online by citizens, public bodies and professionals was one of the objectives of the projects of the Single Programming Document of the Tuscany Region. Within this framework, the provincial authority of Florence supported the creation of a spatial database linked to the theme of hydrogeology, as well as the development of software to manage the administration of the use of ground and surface water. It also sought to manage fully online the submission of groundwater search requests and the issuing of authorisations.

Workflow system — Warmia i Mazury Region, Poland

The main objective of the project was to create a web portal allowing online contact between the public and 112 authorities. Electronic document circulation procedures were developed for three levels of local administration, aiming to improve operational efficiency whilst ensuring full interoperability of data processed by the administration.

OBSERVATIONS

HAVE THE PROJECTS BEEN SELECTED ACCORDING TO ASSESSED NEEDS?

16. The purpose of e-Government is to enhance access to government services and information for the public, businesses and other administrations, leading to better service quality and improved effectiveness and efficiency.

17. Bringing this about requires an overall strategy based upon a needs assessment, the definition of priorities and a consideration of the likely costs and benefits of the strategy.

18. The Court examined whether:

(a) the national or regional strategic framework had been developed on the basis of identified needs and knowledge of what was being provided by existing IT systems, taking into account guidance issued by the European Commission;

(b) project selection procedures were based upon consistent and coherent selection criteria which addressed the most urgent needs and delivered added value for stakeholders; with individual project objectives complying with the priorities of the strategic framework.

INITIAL STRATEGIES WERE NOT BASED UPON IDENTIFIED NEEDS

19. In the 2000–06 period communication patterns evolved from those based upon telephone and paper to an extensive use of the Internet and broadband. A number of countries began to adapt their administrations accordingly.

20. In the European Union, the eEurope Action Plans prioritised e-Government related initiatives, which linked regional competitiveness and effective public administration. Throughout the period, Member States committed themselves, by way of ministerial declarations to proposing relevant actions at national level, leading to the preparation of national strategies in line with the eEurope initiative.

21. At the beginning of the programme period, the strategies for e-Government in the Member States visited had been prepared in terms of broad declarations, rather than based upon real identified needs.

22. The Court examined the relevant strategy documents for the Member States visited and detected the following weaknesses:

(a) The needs of citizens, businesses and administration were not determined in advance and strategic objectives were too general and lacked specific targets. There was insufficient analysis of what was actually required. For example, in certain countries, the necessary legal provisions for allowing electronic transactions were not in place.

(b) Responsibilities for the achievement of objectives had not been clearly attributed to the responsible public bodies (see Box 3) or these bodies lacked sufficient authority for effective implementation. Monitoring systems were not established thus preventing timely corrective actions being taken and the accumulation of experience for future improvements.

BOX 3

EXAMPLE OF UNCLEAR ASSIGNMENT OF RESPONSIBILITIES

Strategy "e-Poland for the years 2004–06"

The main weakness of the strategy was a failure in all cases to clearly define areas of responsibility. There were 83 specific tasks allocated to various ministries and agencies and 30 of these had more than one entity responsible for its implementation. However in these cases, no indication had been given as to who was responsible for coordination. Insufficient coordination and cooperation amongst ministries was highlighted in the final strategy evaluation as being one of the factors preventing the achievement of objectives.

23. A comparison of early strategies with those prepared at the end of the audited period clearly indicates some improvement in quality. In addition, some countries prioritised e-Government, recognising it as a way to modernise public administration. However at Member State level there was little methodological support offered to the authorities responsible for project delivery, in particular as regards project management techniques and technical guidelines. See Box 4 for an example of methodological support which was actually given.

BOX 4

AN EXAMPLE OF METHODOLOGICAL SUPPORT

The Italian National Centre for the Information Systems in the Public Sector (CNIPA) was established in June 2003 as a public agency under the aegis of the Italian Ministry of Public Administrations and Innovations.

CNIPA became responsible for putting into operation the e-Government plans devised by the ministry and were specifically charged with:

(a) enacting rules, standards and technical guidelines for the introduction of ICT within the public administration;

(b) assessing and monitoring projects developed by local, regional and national administrations;

(c) drafting opinions for the public administration on the strategic coherence of projects.

PROJECT SELECTION RARELY ASSESSED WHETHER THE PROJECTS SELECTED WERE THE MOST LIKELY TO OFFER THE BEST VALUE FOR MONEY

24. The project selection procedures were organised and managed by national authorities responsible for ERDF management and usually followed one of the models below:

(a) Model A — Projects were preselected, named and their scope indicated in the ERDF programming documents (e.g. programme complement).

(b) Model B — Following a call for proposals, projects were selected on a competitive basis, with grants being offered to projects best fulfilling measure objectives. Specific amounts of money were made available for such measures, as the development of electronic public administration.

25. The Court analysed 28 projects selected on the basis of the two procedures (11 A, 17 B):

(a) The centralised approach (Model A), with ERDF grants being assigned in advance to named projects. There was little attention paid to any analysis of costs or benefits and quite often there was no formal grant application procedure.

(b) The other approach (Model B), designed to select the best projects. This addressed priority issues defined in the programming documents, on the basis of project content comparison carried out by independent experts. "Cost benefit analysis" was an obligatory part of the application procedure in Poland. The bodies responsible for selection did not set out a timetable for the overall procedure other than the deadline for applications. When there were many applicants, there were long delays in the assessment of applications and in Poland the original start date of four of the projects visited had to be postponed due to the longer selection procedure.

26. Authorities sometimes tended to select ambitious projects in terms of goals, scope and estimates of the number of potential users, whilst not sufficiently assessing the promoters’ ability to manage complex IT projects or establish if they had sufficient and appropriately qualified staff. In one project in Poland, the first project manager was appointed in 2006, 17 months after the financing was granted. During 2007, five different project managers were responsible for the task, whilst the composition of the steering committee overseeing the project was amended four times. This lack of stability was cited in an internal audit report as being partly responsible for project delays. In a final assessment report on an Italian project, delays in implementation were attributed to changes in the project management team and a lack of coordination between local bodies participating in the project.

27. In fact, most projects which failed to meet their original objectives were very complex as regards the number of stakeholders involved and the sophistication of services offered. This situation, in combination with the need to finish projects before the closure of the programme caused a significant scope reduction in some cases (see paragraph 39).

WERE PROJECTS WELL DESIGNED AND DELIVERED AS PLANNED?

28. An e-Government project, as with any IT project, should be planned in detail to ensure a smooth realisation. The plan should include clear objectives with a detailed description of the actions necessary for project completion, and an estimate of time and resource constraints.

29. The aim of the Court was to assess whether projects were carried out as initially planned and to identify the reasons for major deviations. For each audited project the fundamental project elements were examined:

(a) Planning and design — Special attention was paid to the appropriate setting of project objectives and targets in the form of SMART indicators and the use of best practice, such as the use of recognised project management methodology (PMM).

(b) Compliance with European interoperability framework (EIF) principles — The incorporation in project design of EIF principles was examined.

(c) Implementation — The process was examined in terms of the management of time, budget and project scope.

(d) Availability for use — The existence of project assets (e.g. IT software, servers, platforms) as well as planned services was checked during on-the-spot inspections.

PROJECT OBJECTIVES WERE TOO GENERAL AND PROJECT DESIGN DID NOT DEAL ADEQUATELY WITH NON-TECHNICAL ISSUES

30. The setting of detailed project objectives was a necessary pre-requisite for public bodies applying for an ERDF grant. However projects were normally defined in general terms in relation to national or regional strategies and objectives contained in the ERDF programming documents. Project added value was rarely determined as apart from the projects in Poland, cost benefit analyses were not prepared.

31. The quality of project documentation varied significantly from project to project and in extreme cases no documentation was available for examination. The responsible authorities had not set out minimum standards in terms of documentation amongst other things.

32. For the majority of projects visited, objectives were set which were too general and not quantified. For instance, setting objectives as vague as "to improve service quality". Moreover where targets were set, they referred mainly to outputs (e.g. number of servers purchased), rather than focusing on benefits for the entity or end-user (e.g. decrease the running cost of the department by x %). According to the project promoters surveyed (Figure 2) impact indicators were defined for only 42 % of projects.

FIGURE 2

WHAT TYPE OF PROJECT OBJECTIVES WERE SET?

+++++ TIFF +++++

The output and result objectives were clearly set in most of the cases, but objectives in terms of impact were less often established.

Source: European Court of Auditors’ survey.

33. Project planning was strongly focused on technical aspects, often neglecting important issues such as the redesign of administrative processes to benefit more fully from the new technologies and thereby lessening the potential impact of projects.

34. Failure to consider coordination between departments or organisational issues such as the reconfiguration of processes and workflows for different projects resulted in the duplication of procedures, project delays, problems with data exchange and in some cases projects which were not developed as expected, underused or even abandoned (see Boxes 5, 7 and 9).

PROJECTS MOSTLY IN LINE WITH EIF PRINCIPLES

35. The European interoperability framework (EIF) was published in November 2004 and defines a set of guidelines for e-Government services so that public administrations, enterprises and citizens can interact across borders in a pan-European context. The 15 projects visited which started in 2005 or later were examined to see to what extent they had taken into account these guidelines.

BOX 5

EXAMPLE OF INADEQUATE COORDINATION AND ITS IMPACT

Integrated broadband platform for Sicily, Italy

One of the objectives of this project was the development of a web portal for tourists with specific interactive features such as booking and paying for holiday packages online.

The lack of coordination between different directorates within the regional administration caused significant delays and scope reduction with some planned features not being developed.

36. The Court assessed these projects for compliance with the principles of:

(a) Accessibility — Access was mainly through the Internet and in a few cases other channels of communication, such as mobile phone, were used. Most projects used recommended guidelines [17] for the design of web pages and interfaces.

(b) Multilingualism — The front office services were mainly limited to the national language(s) and only 17 % of them provided information in more than one language.

(c) Security and privacy — The Court found that due consideration had been given to the problems of security and privacy of data and there was a proper level of security and compliance with the relevant legislation.

(d) Use of open standards and open software — The use of open standards is common and many projects could produce and exchange data in various open formats. Open software is frequently used as it allows IT contractors to offer a more competitive price. The systems examined were developed using a mixture of commercial and open source solutions.

MOST PROJECTS NOT IN LINE WITH PLANNING

37. The Court examined the project documentation of all 28 visited projects, assessing project progress compared to planning, implementation procedures and the quality of project management.

38. The vast majority of audited projects delivered their outputs as planned; however, 82 % (23) were not implemented in line with the planning defined in the grant application (Figure 3). The main reasons for deviations were as follows:

(a) Planning and design — As a consequence of inadequate or imprecise planning, issues such as the clarification of objectives and responsibilities or the recruitment of supplementary staff, had to be dealt with during implementation.

(b) Procedural — Certain procedures were more time-consuming than foreseen in the original planning, most commonly issues related to procurement, or coordination. There were also general administrative delays linked to bureaucratic procedures, the insufficient empowerment of project teams or legal issues.

FIGURE 3

WAS THE PROJECT DELIVERED ON TIME?

+++++ TIFF +++++

Only 35 % of projects were completed on time, and 17 % were delayed by more than one year.

Source: European Court of Auditors’ survey.

(c) Human resources — Of the small number of public administrations who decided to develop systems themselves, two lacked staff experienced in IT project management resulting in project delays and a high turnover of key personnel.

(d) Exogenous — Factors having an impact on deadlines which were not controllable by project beneficiaries (e.g. behaviour of contractors and their ability to deliver products, changes to the national legal framework).

39. The majority of delays were of less than one year and did not affect project deliverables or markedly increase overall costs, as most contracts included a fixed-price clause which protected against cost increases. However, delays of more than one year combined with the need to finish projects before the closure of the programme had an adverse impact on three projects leading to a reduction in scope (see Box 6).

40. Of the 28 projects visited 10 (36 %) used a project management methodology. Results from the survey revealed that only 23 % of projects had been implemented using PMM principles. The Court found that 50 % of visited projects which had used a recognised PMM had been completed on time and 90 % within budget. In the 18 projects where PMM was not used, only 22 % were on time and 56 % had kept within budget.

BOX 6

EXAMPLE OF SCOPE REDUCTION RESULTING FROM DELAYS

Electronic office — Online procedures in the Canary Islands, Spain

The rules for the 2000–06 programme period required all ERDF co-financed projects to be completed before the end of 2008. Due to poor planning, the project experienced a delay of five years, posing a risk that the ERDF grant would have to be returned. To avoid the loss of EU co-financing, the original scope of the project had to be significantly modified. Although they had intended to include over 50 procedures for electronic processing, the project managers decided to focus on the main two, accounting for approximately 75 % of the total number of files processed.

41. In one such case, insufficient focus on organisational issues meant that the IT system that had been developed could not be used. The contracts necessary for obtaining the data to be input to the system had not been signed in due time and therefore a technically operational system was redundant.

42. Another factor having an impact on the use of systems resulting from e-Government projects was the commitment of management to exploit the new solutions and services. These systems often offered new functions beyond those traditionally provided by public administrations, such as electronic tendering or personalised user profiles for members of the public. Such features constitute the main added value of new electronic systems and enable the modernisation of public administration. However in some cases the benefits arising from new technology were unrealised (see Box 7).

BOX 7

EXAMPLE OF UNUSED FUNCTIONS OF IT SYSTEMS

Internet portal of the Maritime Office in Gdynia, Poland

The computerised system was designed to provide online access to services for customers of the Maritime Office. The system offered a procurement module which made electronic tendering possible. A feasibility study estimated positive benefits from the use of electronic tendering and an objective of organising 30 online auctions per year was set but despite full system implementation by the time of the audit visit, no Internet auctions had taken place.

ARE THE PROJECTS USEFUL AND DURABLE?

43. Maintenance of an IT product refers to the processes needed to sustain it throughout its operational life cycle including training for key staff and actions necessary to provide support for users, such as the provision of a help-desk.

44. For this part of its audit the Court focused on the following issues:

(a) Usefulness — The final project stage was examined, focusing on how useful the project outputs were, with an assessment of the actual take-up of new services and the related economic benefits yielded by the project.

(b) Maintenance — An assessment of appropriate project maintenance in the foreseeable future without EU intervention and an evaluation of the financial and operational sustainability of projects, taking maintenance into account.

EXPECTED ADVANTAGES PARTIALLY ACHIEVED

45. Almost all audited projects [18] were put into operation shortly after completion, replacing and digitising many existing internal procedures. Despite the fact that outputs were achieved projects were less successful at delivering improved results or positive impacts (Figure 4).

FIGURE 4

HAVE ALL PROJECT OBJECTIVES PRESENTED IN THE APPLICATION BEEN ACHIEVED?

+++++ TIFF +++++

In general objectives linked to outputs were met, but there was less success with regard to results and especially impact objectives.

Source: European Court of Auditors’ survey.

46. It is clear that there is interest within public administrations for the development of e-Government services but the take-up on the part of business or the general public has remained below the estimates contained in the project applications. The Court considers that there are a number of reasons cited by project promoters when interviewed which could explain this:

(a) Electronic identification — Most services carried out by public administrations require identification. For digital services electronic identification is normally required but its level of use varies from country to country. In certain Member States such as Spain, the electronic signature is embedded in the national identification card, therefore access to identification card readers can become a barrier to its use. Whereas in Poland, for example, the e-signature is sold as a separate certificate and therefore cost can limit access to digital services.

(b) Legal framework — Paper-based transactions are embedded in our culture of administration. There are still executive regulations requiring the presentation of original paper documents, making impossible the full electronic handling of applications and lessening the potential benefits from electronic communication.

(c) Access — Although not a problem in some countries and in urban areas, lack of access to computers and broadband networks still impacts on service take-up. Computer literacy can also be a problem affecting demand for electronic services.

47. According to the project promoters interviewed, the benefits are particularly noticeable in organisations where IT systems replaced a significant part of their standard processes and where they are used by the majority of operational departments. The benefits stated were the following (Figure 5):

(a) Simplification of procedures — IT systems allow for single-access or contact points where requests are automatically directed to the appropriate personnel within an organisation and this makes access more straightforward for the end-user.

(b) Better case monitoring — More sophisticated back office systems are a powerful monitoring tool for public bodies. Better information about workloads and available resources allows for more accurate planning and better use of human resources.

(c) Time savings — Access to centralised files and digital documents allows quicker response times and eliminates time lag linked to document circulation. It also facilitates access to both current and archived files (see Box 8).

(d) New channels of communication — The Internet and e-mail provided new channels of communication for the exchange of information and allowed the public to access services in a more convenient and user-friendly way.

BOX 8

EXAMPLE OF EFFICIENCY GAINS

Land information system for the city of Łódź — Phase V, Poland

This project consisted of the building of a modern portal and system of databases of geographical information, to provide online land and property information on the city of Łódź.

According to calculations made by management, the project has resulted in a considerable increase in efficiency (20 % to 52 %, across the various departments), estimated at having a value of approximately 4,5 million PLN in 2009 (ca. 1,1 million euro). This was in excess of the 3,2 million PLN (ca. 0,8 million euro) estimated in the Feasibility Study for 2009.

48. However the Court considered that in general insufficient attention had been paid to possible project benefits and the advantages offered by advanced technology were not fully exploited. In one Italian project, the software developed was used for internal administrative purposes only although the planned objective of the project was to extend the electronic service to the end-user. Moreover, certain projects visited in France and Poland kept paper-based processes running in parallel with IT workflow systems with no defined end-date, leading to duplication and reducing the potential benefits of the IT development.

FIGURE 5

WHAT WERE THE BENEFITS FROM THE PROJECT AND FOR WHOM?

+++++ TIFF +++++

In the opinion of project managers, the projects delivered a wide range of benefits for users.

Source: European Court of Auditors’ survey.

49. Globally in the absence of project monitoring or measurement of the actual benefits, it is not possible to determine to what extent new services increased the quality of public administration. There was only one project in the sample which could quantify the tangible benefits resulting from the aid granted. Without such measurement, valuable feedback knowledge is not being built up and the experience gained from these projects, which could lead to better planning of future projects, is being lost.

IN MOST CASES PROJECT MAINTENANCE WAS ASSURED

50. A significant majority of the projects ensured proper technical and organisational (e.g. help-desk) maintenance for the systems developed. Usually maintenance and upgrading were carried out by contractors during the warranty period which lasted from one to three years. After this period, the IT systems were supervised by in-house IT Departments or the maintenance was outsourced, depending on the availability of IT staff in-house. The result of the survey provides a similar view to that from the projects visited (Figure 6).

FIGURE 6

IS A MAINTENANCE CONTRACT IN PLACE FOR YOUR IT PRODUCT?

+++++ TIFF +++++

The maintenance for 86 % of the projects was assured by internal IT Departments or outsourced experts.

Source: European Court of Auditors’ survey.

51. Where planning of the full project life cycle was included in a "Cost benefit analysis", maintenance costs were budgeted for. In those cases there was little risk for project continuity as beneficiaries had anticipated the resources required. However in three cases, underestimation of maintenance costs posed a risk to projects being put into use or continuing. Failure to factor in maintenance costs meant that beneficiaries were unable to finance operational costs in whole or in part and in one case, caused the project to shutdown after the warranty period (see Box 9).

BOX 9

IMPACT OF MAINTENANCE COSTS ON PROJECT SUSTAINABILITY

e-Government: centre for services to citizens and businesses and infrastructure services in Sicily, Italy

The aim of the project was to establish a technological infrastructure, shared by 57 participating bodies, and to implement a number of IT applications for the joint management of a raft of e-Government services.

The system was developed and became operational in April 2007. As the maintenance costs and its allocation between the participating bodies had not been foreseen at the planning stage, in May 2008, the application was shut down.

CONCLUSIONS AND RECOMMENDATIONS

52. The Court concluded that the e-Government projects supported by the ERDF have contributed to the development of electronic public services in the four selected Member States. However, despite the fact that the majority of projects are technically operational, insufficient focus on project results has meant that the benefits obtained were much lower than expected.

53. The recommendations below have relevance for the European Commission as responsible for the execution of the budget and for the application of the principle of sound financial management and Member States’ managing authorities in the context of e-Government actions for the remainder of the 2007–13 programme period and for the design of future e-Government or similar schemes.

54. The Court’s conclusions and recommendations related to each of the audit subquestions are as follows:

HAVE THE CO-FINANCED PROJECTS BEEN SELECTED ACCORDING TO ASSESSED NEEDS?

55. Despite the fact that e-Government programming at national level gradually improved during the 2000–06 programme period, weaknesses in early strategies meant that the co-funded projects did not focus on addressing priority needs for e-Government development.

56. This is mainly due to the following:

(a) Early e-Government strategies were prepared mainly in response to political declarations instead of a rigorous needs assessment. As a result strategies and their objectives were unable to address the most important and urgent issues which could unlock and boost e-Government development (see paragraph 20).

(b) When projects were preselected centrally, there was little or no evaluation of the costs or benefits of projects or their potential impacts and results (see paragraph 25).

(c) In cases where insufficient consideration was given to the capacity of project teams, in terms of skills and resources, to deliver e-Government systems as planned, this resulted in significant scope reductions (see paragraph 26).

RECOMMENDATION 1

(a) Member States should develop strategies for e-Government, which are based upon identified needs, have clear objectives and assign responsibilities to the bodies accountable for the achievement of these objectives.

(b) Managing authorities should select e-Government projects for ERDF support on the basis of an assessment of likely project costs and benefits.

WERE PROJECTS WELL DESIGNED AND DELIVERED AS PLANNED?

57. The projects audited achieved results in the areas of increasing the availability of electronic public services and in the development of technical means within public authorities.

58. Project outputs, although generally delivered, were often delivered late or with a reduction in scope due to poor design or lack of an implementation methodology. The main identified systemic problems were as follows:

(a) Most project promoters failed to identify and quantify project benefits and as a result objectives were mainly set in terms of outputs. In consequence, project benefits (see paragraph 32) were rarely completely achieved and the possibilities given by new technologies were not exploited to the full. This reduced the potential benefits of projects for end-users (see paragraph 46). Organisational and administrative issues, when addressed, proved more difficult to resolve than technical problems (see paragraph 33 and 34).

(b) In many cases, the lack of a project management methodology in addition to inadequate planning impacted negatively on project implementation, causing delays or scope modifications. Visited projects which used PMM performed better in terms of being completed on time and on budget (see paragraph 40).

(c) Most of the EIF principles were well addressed in the design of projects, however consideration should be given to the provision of services in more than one language when this is of assistance to significant user groups (see paragraph 36).

RECOMMENDATION 2

(a) Managing authorities in Member States should ensure that e-Government projects selected for ERDF funding focus not only on project outputs but also on the changes in processes or organisation necessary to fully benefit from the systems developed.

(b) Managing authorities should emphasise the practical application of best practice and strongly recommend the use of an appropriate project management methodology for e-Government projects in receipt of ERDF funding.

(c) The Commission should ensure that the EU principles and recommendations enabling trans-European interoperability, in particular the EIF principles, are considered by projects benefiting from the ERDF, in order to increase the European added value of a project and facilitate further EU-wide systems integration.

ARE THE PROJECTS USEFUL AND DURABLE?

59. Most of the projects audited were technologically sound and the IT applications developed provided electronic services to public bodies, citizens and businesses. In general, the systems co-financed by the ERDF, were appropriately maintained and financially sustainable.

60. However, many projects did not deliver all of the expected benefits to end-users and indeed the quantification of results, in any precise way, was usually not possible. This situation was mainly due to the following:

(a) Organisations rarely reconfigured their processes or structures to take optimum advantage of the possibilities created by the new systems. In some cases extra work was created by running paper and electronic versions of the same process in parallel (see paragraph 48).

(b) Underestimation of maintenance costs or the failure to take them into account at all put a financial strain on projects and in three cases posed a risk to their continued operation (see paragraph 51).

(c) There was little or no project follow-up providing management information related to project results. The lack of such data makes proper project evaluation impossible and prevents the accumulation of experience which can lead to better e-Government project design in the future (see paragraph 49).

RECOMMENDATION 3

(a) When selecting e-Government projects, managing authorities should ensure that all significant costs, including maintenance have been sufficiently provided for in the cost-benefit analysis which should underpin the financing decision.

(b) The Commission should ensure that managing authorities monitor and evaluate project results and impacts in order to demonstrate the effective use of EU funding and provide important feedback to improve the design of future programmes.

This report was adopted by Chamber II, headed by Mr Morten LEVYSOHN, Member of the Court of Auditors, in Luxembourg at its meeting of 13 July 2011.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] The role of e-Government for Europe’s Future, COM(2003) 567 final of 26.9.2003, p. 7.

[2] For example lack of interoperability between IT systems could be a barrier hampering exchange of administrative data.

[3] The European interoperability framework was endorsed in June 2002 by the 15 Member States and was supported in the Manchester and Lisbon declarations of 24 November 2005 and 19 September 2007, respectively.

[4] http://ec.europa.eu/information_society/eeurope/i2010/benchmarking/index_en.htm

[5] eEurope 2002: Impact and Priorities, COM(2001) 140 final of 13.3.2001.

[6] COM(2003) 567 final.

[7] i2010 – A European Information Society for growth and employment, COM(2005) 229 final of 1.6.2005.

[8] World Economic Forum Global Competitiveness Reports - (http://www.weforum.org/reports-results?fq=report%5Ereport_type%3A%22Competitiveness%22), Citizen Advantage: Enhancing economic competitiveness through e-Government (http://unpan1.un.org/intradoc/groups/public/documents/apcity/unpan022639.pdf)

[9] i2010 e-Government Action Plan: Accelerating e-Government in Europe for the benefit of all, COM(2006) 173 final of 25.4.2006.

[10] Ministerial Declarations approved unanimously in Brussels (2001), Como (2003), Manchester (2005) and Lisbon (2007).

[11] Directive 2006/123/EC of the European Parliament and of the Council of 12 December 2006 on services in the internal market (OJ L 376, 27.12.2006, p. 36).

[12] Programmes such as Modinis, eTEN, IDABC, CIP-PSP, ePractice.eu.

[13] Decision 2004/387/EC of the European Parliament and of the Council of 21 April 2004 on interoperable delivery of pan-European e-Government services to public administrations, business and citizens (IDABC) (OJ L 181, 18.5.2004, p. 25). The programme expired in 2009.

[14] While the IDABC was more focused on the digitisation of public administrations, its follow-on ISA aims at elaborating concrete generic tools and common services which could be directly applied by interested Member States. Decision No 922/2009/EC of the European Parliament and of the Council of 16 September 2009 on interoperability solutions for European public administrations (ISA).

[15] The Court calculation made on the basis of information published by the Commission on the http://www.europa.eu

[16] For the 2000–06 programme period, the Structural Funds and European Regional Development Fund was funded on the basis of the following legal acts: Council Regulation (EC) No 1260/1999 of 21 June 1999 laying down general provisions on the Structural Funds (OJ L 161, 26.6.1999, p. 1) and Regulation (EC) No 1783/1999 of the European Parliament and of the Council of 12 July 1999 on the European Regional Development Fund (OJ L 213, 13.8.1999, p. 1).

[17] The World Wide Web Consortium (W3C) is an international community that develops standards to ensure the long-term growth of the web. (World Wide Web Consortium (W3C)).

[18] The only exception were two cases where due to coordination issues and lack of project ownership the use of systems was indefinitely postponed or completely abandoned.

--------------------------------------------------