EUROPEAN COMMISSION

Brussels, 17.12.2021

COM(2021) 965 final

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Results of the final evaluation of the ISA² programme

{SWD(2021) 965}

Table of contents

1.    Introduction    

2.    What was expected from the ISA2 programme?    

3.    How has the situation evolved during the evaluation period?    

3.1.    Adoption of a new EIF    

3.2.    Political context    

3.3.    An increasingly complex EU regulatory environment    

3.4.    Digitalisation investments    

4.    Evaluation findings    

4.1.    To what extent was the ISA2 programme successful?    

4.2.    How did the ISA2 programme make a difference to the lives of citizens and businesses?    

4.3.    Are the ISA2 programme and its objectives still relevant?    

5.    Conclusions and lessons learned    

5.1.    Importance of interoperability for the implementation of EU policies    

5.2.    From a coherent to a strategic approach to EU public sector interoperability    

5.3.    Development and take-up of mature and user-friendly interoperability solutions    

6.    Next steps    

1.Introduction

The ISA2 Decision 1 on 1 January 2016 launched the five-year programme on interoperability solutions and common frameworks for European public administrations, businesses and citizens (ISA2 programme). It is the fifth in a series of European Commission programmes 2  focusing on interoperable solutions for the public sector. Interoperability as defined in the European Interoperability Framework (EIF) is not a technical issue. It is more about legal, organisational, semantic and technical requirements to make integrated public services possible.

This report presents the findings and recommendations of the final evaluation of the ISA2 programme 3 . Under Article 13(3) of the ISA2 Decision, the Commission must carry out this evaluation and communicate its results to the European Parliament and the Council by 31 December 2021.

The Commission carried out the evaluation using a team of independent experts from a consultancy 4 . The evaluation was overseen by an interservice steering group 5 and carried out in close coordination with the evaluation of the EIF and impact assessment for a future interoperability policy 6 , which the Commission plans to propose to the Parliament and Council in Q2 2022.

2.What was expected from the ISA2 programme?

The ISA2 programme aimed to achieve the following four objectives (Article 1(1) ISA2 Decision1):

(a)Develop, maintain and promote a holistic EU approach to interoperability.

(b)Facilitate efficient and effective electronic cross-border or cross-sector interaction to contribute to the development of a more effective, simplified and user-friendly e-administration at national, regional and local levels of public administration.

(c)Identify, create and operate interoperability solutions supporting the implementation of EU policies and activities.

(d)Facilitate the re-use of interoperability solutions by European public administrations.

To achieve these objectives, the programme built on the achievements of its predecessor, the ISA programme 7 . It also put emphasis on fitting smoothly into the wider policy framework for digitalisation, particularly on the interoperability of public administrations in the EU. Article 1(2) of the ISA2 Decision states that the ISA2 programme shall ensure a common understanding of interoperability through the European Interoperability Framework (EIF) and its implementation in Member States’ administrations. 

Other policy initiatives to which ISA2 was supposed to contribute include the eGovernment Action Plan 2016-2020 8 as well as several sector-specific interoperability initiatives (e.g. in the area of justice and home affairs, health, European funds) 9 .

ISA2 ran from 1 January 2016 until 31 December 2020 with a total budget of EUR 130.9 million. Some activities are still ongoing. It funded actions defined on a yearly basis in the annual rolling work programme. ISA2 was open to EU Member States, other members of the European Economic Area and candidate countries. In addition to the 28 EU Member States (27 for 2020), four other countries took part in the programme 10 .

3.How has the situation evolved during the evaluation period?

During the evaluation period, digitalisation and more precisely interoperability have become more important at all levels of society. As a result, it is a continued priority of the European Commission.

3.1.Adoption of a new EIF

As planned in the Communication on a Digital Single Market Strategy for Europe 11 and the eGovernment Action Plan 2016-2020, the EIF was updated and extended in 2017. The new EIF was set out in a communication 12  and accompanied by an interoperability action plan. The new EIF gave the ISA2 programme a new policy framework consistent with the programme’s objectives. In February 2020, the Commission announced in its Communication ‘A European strategy for data ’ 13  that it would present a strengthened EIF.

3.2.Political context

During the evaluation period, the political support from Member States has been steadily growing. With the Tallinn Declaration on eGovernment 14 , the ministers in charge of eGovernment policy across the EU spelled out their commitment to several principles, including ‘interoperability by default’. This commitment was recently renewed by the Berlin Declaration on Digital Society and Value-Based Digital Government 15 , which identified the need to take a value-based approach – incorporating digital sovereignty and interoperability – to the digital transformation of the public sector.

In February 2020, the Commission adopted the Communication ‘ Shaping Europe’s digital future ’ 16 under the headline ambition ‘Europe fit for the digital age’. It sets out as a key action the development of a ‘reinforced EU governments interoperability strategy’, which aims to improve coordination and the adoption of common standards for public services and data flows.

3.3.An increasingly complex EU regulatory environment

The ISA2 Decision refers to a wide range of sector-specific and cross-cutting EU interoperability activities at EU level. Further key EU initiatives have been adopted since the programme began:

·2016 – the General Data Protection Regulation 17 and the Web Accessibility Directive 18  came into force.

·2018 – the Regulation on the  Single Digital Gateway 19  came into force, as did the Regulation on the free flow of non-personal data 20 .

·2019 – the EU Cybersecurity Act 21  and other related legislation came into force as well as the Open Data Directive 22 .

·2020 – the Commission proposed the Digital Services Act 23 , the  Data Governance Act 24  and, in response to the COVID-19 crisis, interoperability initiatives around eHealth 25 .

3.4.Digitalisation investments

Public administrations across the EU are investing heavily in digitalising their services and back-offices. The COVID-19 crisis has highlighted the importance of digital solutions even more. Besides ISA2, other EU programmes contributed to this digitalisation drive during the evaluation period:

·The Structural Reform Support Programme 26 , implemented from 2017 to 2020, followed by the Technical Support Instrument for 2021 to 2027 27 .

· Horizon 2020 28 (dedicated to research and innovation), the European Social Fund 29 and the European Regional Development Fund 30  funded digitalisation. The Recovery and Resilience Facility 31  will provide another important source of funding in the future, as 20% of its budget must be allocated to digital expenditure. 

4.Evaluation findings

The evaluation of the ISA2 programme relied on quantitative and qualitative data collected using various methods. This included desk research, public consultation, targeted online surveys, in-depth interviews and two stakeholder workshops. The consultation activities allowed for a broad coverage of ISA2 stakeholders – from representatives of EU Member States and Commission services to citizens and standardisation bodies.

For better data collection, a sample of 21 actions was selected from the 54 actions – grouped into 9 packages – funded by the ISA2 programme. Three pre-defined criteria were used for the sampling to ensure that the selected actions are largely representative of the programme. The overall data collection was also complemented by an expert assessment of the programme carried out by five technical interoperability experts.

The evaluation focused on the following seven main criteria 32 : relevance, effectiveness, efficiency, coherence, EU added value, utility and sustainability.

4.1.To what extent was the ISA2 programme successful?

Based on data collected from 102 consulted stakeholders 33 , extensive desk research, and expert assessments, the evaluation shows that the ISA2 programme has gone some way towards achieving its objectives. Consulted ISA2 solution users and action owners, who are more knowledgeable about the programme, generally consider that it has gone a considerable way towards achieving its objectives. In general, out of all consulted stakeholder groups, respondents from national and sub-national public authorities gave the lowest scores in terms of the programme achieving its objectives.

On the general and specific objectives, most respondents and desk research confirm that the following objectives have been achieved to some extent or to a great extent:

·‘Identifying, creating, and operating interoperability solutions supporting the implementation of EU policies and actions’ (overall average score of 3.43 out of 5). ISA2 actions have contributed in various ways to the implementation of the EIF 34 , the Single Digital Gateway 35 , the Open Data Directive 36 ,  e-procurement 37 , the Inspire Directive 38 , the European Statistical System 39 and  financial legislative acts 40 .

·‘Facilitating the re-use of interoperability solutions’ (overall average score of 3.43 out of 5). The Joinup action 41 , the platform with the same name, provides a comprehensive overview of existing interoperable solutions available for re-use – including access to several national and even local portals. The interactive map 42  on the ISA2 webpage shows the take-up of ISA2 solutions in the EU. More could be done to boost their take-up and encourage their re-use, especially regionally and locally.

·‘Developing, maintaining and promoting a holistic approach to interoperability in the EU’ (overall average score of 3.39 out of 5). ISA2 plays an important part in raising awareness of interoperability, having set up structures for cooperation with EU countries (the ISA2 Committee) and supporting the promotion and monitoring of EIF implementation. 

The ISA2 programme has been less successful in contributing to ‘developing more effective, simplified and user-friendly public e-administration at the national, regional and local levels’ (overall average score of 3.20 out of 5). The evaluation showed that there is still limited knowledge and take-up of solutions regionally and locally. There is also the lack of a strategic and holistic approach.

Work across ISA2 packages has progressed as expected relative to the planned work and budget. The heterogeneity of performance indicators made it difficult to draw conclusions about the programme’s overall cost-effectiveness. For those packages where it was possible to apply cost-effectiveness techniques, costs per end-user (e.g. business, citizens, etc.) have been estimated as low. The benefits are therefore greater than the costs for the ultimate beneficiaries of the solutions developed. However, benefits risk fading in the absence of appropriate sustainability measures.

Substantial synergies and limited overlaps among ISA2 actions paint a positive picture of the programme’s internal coherence. However, prioritising a smaller set of actions would increase synergies. ISA2 worked closely with a wide range of other relevant EU programmes, policies and initiatives. This could, however, be done more systematically.

4.2.How did the ISA2 programme make a difference to the lives of citizens and businesses?

ISA2 aimed to make a difference to the lives of citizens and businesses by helping EU public administrations reduce the administrative burden and provide user-friendly public services, especially when interacting with public administrations in other EU countries. The Commission and the European Parliament have acknowledged that small and medium-sized enterprises in particular suffer from administrative burden 43 . End-users, citizens and businesses should benefit from common, reusable and interoperable front office services resulting from the better integration of processes and exchange of data through the back offices of public administrations in the EU.

ISA2 has contributed to EU-level coordination, necessary for improving overall interoperability among European public administrations – including front- and back office solutions. Most respondents confirmed that national or sub-national initiatives alone would have only gone a small way towards achieving most of the objectives. Consulted solution users, national and sub-national public administrations and stakeholders responsible for programme governance themselves consider that an EU-level approach is therefore better suited to addressing the challenges related to the interoperability of public administrations and services. ISA2 could also achieve its objectives at a lower cost than comparable national or sub-national initiatives. However, according to most respondents across stakeholder groups, when it comes to reducing the cost and administrative burden of cross-border interaction, such objectives have been achieved only to a limited extent.

The way solutions meet user needs still needs to improve. These needs and problems are expected to be better addressed due to more extensive take-up of ISA2 solutions across European public administrations.

To sum up, the programme has made limited tangible differences to the lives of citizens and businesses due to its technical nature. Nevertheless, it has brought EU added value. The evaluation suggests that its benefits could become sustainable by further developing some of its solutions in open-source communities or by encouraging companies to develop services using free ISA2 solutions under the European Union Public Licence 44 .

4.3.Are the ISA2 programme and its objectives still relevant?

The original needs and problems that the programme intended to address remain highly relevant. The COVID-19 pandemic has demonstrated the programme’s relevance and the relevance of interoperability initiatives more generally. The pandemic has also made existing needs more pressing, including the need for coordination when implementing digital solutions at EU level 45 . 

Stakeholders complain that a holistic approach to interoperability is promoted but not ensured, especially when it comes to the interoperability of the solutions provided to implement EU policies.

5.Conclusions and lessons learned

The evaluation paints an overall positive picture of how the ISA2 programme has performed. Public sector interoperability is now more relevant than ever to create added value for citizens and businesses. The programme was implemented in a mostly non-binding policy framework that is evolving at an unprecedented pace (section  3.3 ). The upcoming substantial investments in public sector digitalisation (section  3.4 ) call for close consideration of the lessons to be learned from the ISA2 programme and the need to continue to build on its achievements.

Said lessons are relevant for the continued activities under the Digital Europe Programme 46 and for a future interoperability policy. However, we must not lose sight of the need for an overall approach to EU public sector interoperability across policies and programmes. 

5.1.Importance of interoperability for the implementation of EU policies

ISA2 actions have contributed successfully to the implementation of a wide range of EU policies. A growing number of policies 47 have a significant impact on the digital infrastructure of the EU public sector. The coordination of sector-specific proposals with enhanced interoperability governance is the key to ensuring smooth implementation and interoperable European digital public services. Even where no central solutions are provided, EU Member States have asked for more coordination and exchange of best practices in the digital implementation of EU policies. COVID-19 related interoperability solutions have shown that the EU is increasingly being asked to provide interoperable solutions that countries can reuse or link to.

The impact goes beyond the level of EU Member States. Regional and local public administrations in many countries are at least jointly responsible for providing digital public services. This means that they would benefit from relevant EU policies accompanied by implementation solutions that are interoperable by default.

5.2.From a coherent to a strategic approach to EU public sector interoperability

The evaluation shows that the programme could contribute to the development, maintenance and promotion of a coherent approach to the digital transformation of the EU public sector and to interoperability through the actions related to EIF implementation.

In addition to the general political context (section  3.2 ), the Commission has recently adopted a communication on the 2030 Digital Compass: the European way for the Digital Decade 48 . That document emphasises the need to ‘ensure interoperability across all levels of government and public services’. With the Berlin Declaration, the ministers in charge of eGovernment policy in the EU Member States highlighted the importance of interoperability also for digital sovereignty and as a prerequisite for value-based digitalisation.

In the future, different options should be carefully assessed to address the outstanding issues, taking into account the following needs addressed by stakeholders:

·Need for enhanced interoperability governance involving EU countries and regional and local public administrations.

·Need for digital-ready EU policies.

·Need for structured and proactive collaboration and the exchange of best practices between all levels of EU public administrations (national, regional, local) involved in developing digital public services.

·Need for strategic and collaborative interoperability planning across different policy programmes and funding initiatives in order to steer investments towards a set of key mature solutions based on user needs.

·Need for systematic links with European standardisation work and increased awareness in the public sector at all levels of the key role of standards in supporting the twin digital/green transitions.

·Need for the promotion of successful solutions and continued awareness-raising of interoperability.

5.3.Development and take-up of mature and user-friendly interoperability solutions 

The evaluation shows that the programme was useful in identifying, creating and operating interoperability solutions and promoting their re-use. Some concrete actions should be carefully assessed to strengthen these activities and ensure they are sustainable:

·Focus on developing a smaller set of key mature solutions, continually developing them and encouraging their take-up. Concentrating on critical priorities and user needs could also increase user satisfaction.

·Provide a one-stop shop for mature interoperability solutions to help incorporate them into the existing interoperability ecosystem, give existing solutions more visibility and boost their take-up regionally and locally with the support of EU Member States.

·Improve the quality of existing solutions by better considering user needs and involving users in creating them.

·Support experimentation and innovation by setting up an agile process for developing solutions and leveraging innovation procurement coupled with ‘sandboxing’ for testing solutions and getting better feedback.

·Encourage collaboration with open-source communities for the sustainable and open development of existing and future solutions.

6.Next steps

In the ongoing implementation of the Digital Europe Programme and the preparation of a future interoperability policy, the Commission will pay close attention to the findings and recommendations set out above. It will analyse them in order to ascertain whether the issues raised are genuine and address them, in close cooperation with EU Member States.

(1)

   Decision (EU) 2015/2240 of the European Parliament and of the Council of 25 November 2015 establishing a programme on interoperability solutions and common frameworks for European public administrations, businesses and citizens (ISA2 programme) as a means for modernising the public sector (OJ L 318, 4.12.2015, p. 1).

(2)

   1) Community contribution for telematics interchange of data between administrations ( IDA ), 1995-1997; 2) Second phase of the IDA programme ( IDA II ), 1999-2004; 3) Interoperable delivery of pan-European eGovernment services to public administrations, business and citizens ( IDABC ), 2005-2009; 4) Interoperability solutions for European public administrations ( ISA ), 2010-2015; 5) Interoperability solutions and common frameworks for European public administrations, businesses and citizens ( ISA2 ), 2016-2020.

(3)

   The details of the evaluation process and results together with the supporting evidence are available in Commission staff working document: SWD(2021) 965 final.

(4)

   CEPS (2021): Study supporting the final evaluation of the programme on interoperability solutions for European public administrations, businesses and citizens (ISA²), European Commission. DOI: 10.2799/94683

(5)

   Members of the group were Directorates-General CNECT, DEFIS, DIGIT, ECFIN, EMPL, ENER, ENV, ESTAT, FISMA, GROW, JUST, MOVE, NEAR, OP, REFORM, RTD, SANTE, SG, TAXUD and the JRC.

(6)

   More information on the current status of the initiative: https://ec.europa.eu/isa2/shaping-future-interoperability-policy_en

(7)

   Article 1(3) of Decision (EU) 2015/2240.

(8)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions EU, eGovernment Action Plan 2016-2020. Accelerating the digital transformation of government, COM(2016) 0179 final of 19 April 2016.

(9)

   See Recitals 4 to 19 of Decision (EU) 2015/2240.

(10)

   Iceland, Norway (since 2016), Montenegro (since 2018) and North Macedonia (2020). The programme also encouraged cooperation with other non-EU countries and with international organisations and bodies. In 2018, cooperation agreements were signed with Uruguay and Ukraine.

(11)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, A Digital Single Market Strategy for Europe, COM(2015) 192 final of 6 May 2015.

(12)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, European Interoperability Framework — Implementation Strategy, COM(2017) 134 final of 23 March 2017.

(13)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, A European strategy for data, COM(2020) 66 final of 19 February 2020.

(14)

   Tallinn Declaration on eGovernment, made at a ministerial meeting on 6 October 2017 during the Estonian Presidency.

(15)

   Berlin Declaration on Digital Society and Value-Based Digital Government made at ministerial meeting on 8 December 2020 during the German Presidency.

(16)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, Shaping Europe's digital future, COM(2020) 67 final of 19 February 2020.

(17)

   Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (OJ L 119, 4.5.2016, p. 1).

(18)

   Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies (OJ L 327, 2.12.2016, p. 1).

(19)

   Regulation (EU) 2018/1724 of the European Parliament and of the Council of 2 October 2018 establishing a single digital gateway to provide access to information, to procedures and to assistance and problem-solving services and amending Regulation (EU) No 1024/2012 (OJ L 295, 21.11.2018, p. 1).

(20)

   Regulation (EU) 2018/1807 of the European Parliament and of the Council of 14 November 2018 on a framework for the free flow of non-personal data in the European Union (OJ L 303, 28.11.2018, p. 59).

(21)

   Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification and repealing Regulation (EU) No 526/2013 (OJ L 151, 7.6.2019, p. 15).

(22)

   Directive (EU) 2019/1024 of the European Parliament and of the Council of 20 June 2019 on open data and the re-use of public sector information (OJ L 172, 26.6.2019, p. 56).

(23)

   Proposal for a Regulation of the European Parliament and of the Council on a Single Market for Digital Services (Digital Services Act) and amending Directive 2000/31/EC, COM(2020) 825 final of 15.12.2020.

(24)

   Proposal for a Regulation of the European Parliament and of the Council on European data governance (Data Governance Act), COM(2020) 767 final of 25.11.2020.

(25)

    https://ec.europa.eu/health/ehealth/covid-19_en

(26)

   Regulation (EU) 2017/825 of the European Parliament and of the Council of 17 May 2017 on the establishment of the Structural Reform Support Programme for the period 2017 to 2020 and amending Regulations (EU) No 1303/2013 and (EU) No 1305/2013 (OJ L 129, 19.5.2017, p. 1).

(27)

   Regulation (EU) No 2021/240 of the European Parliament and of the Council of 10 February 2021 establishing a Technical Support Instrument, (OJ L 57, 18.2.2021, p. 1).

(28)

   Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No 1982/2006/EC Text with EEA relevance (OJ L 347, 20.12.2013, p. 104).

(29)

   Regulation (EU) No 1304/2013 of the European Parliament and of the Council of 17 December 2013 on the European Social Fund and repealing Council Regulation (EC) No 1081/2006 (OJ L 347, 20.12.2013, p. 470).

(30)

   Regulation (EU) No 1301/2013 of the European Parliament and of the Council of 17 December 2013 on the European Regional Development Fund and on specific provisions concerning the Investment for growth and jobs goal and repealing Regulation (EC) No 1080/2006 (OJ L 347, 20.12.2013, p. 289).

(31)

   Regulation (EU) 2021/241 of the European Parliament and of the Council of 12 February 2021 establishing the Recovery and Resilience Facility (OJ L 57, 18.2.2021, p. 17).

(32)

   See Tool #47 of the Commission’s Better Regulation Toolbox (as of July 2021).

(33)

   For confidentiality and data protection reasons, the evaluation team asked the action owners to contact their solutions users as intermediaries for consultation activities. This two-step approach may have limited the number of responses received.

(34)

    https://ec.europa.eu/isa2/actions/continuously-updating-european-interoperability-strategy_en

(35)

    https://ec.europa.eu/isa2/actions/common-architecture-single-digital-gateway_en

(36)

    https://ec.europa.eu/isa2/solutions/dcat-application-profile-data-portals-europe_en

(37)

    https://ec.europa.eu/isa2/actions/simplifying-public-tenders_en

(38)

    https://ec.europa.eu/isa2/actions/elise_en

(39)

    https://ec.europa.eu/isa2/actions/sharing-statistical-services-and-solutions_en

(40)

    https://ec.europa.eu/isa2/actions/towards-better-financial-data-reporting_en

(41)

    https://ec.europa.eu/isa2/actions/platform-sharing-knowledge-good-practices-and-it-solutions-public-sector_en

(42)

    https://ec.europa.eu/isa2/solutions_en

(43)

    https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52020DC0103   https://www.europarl.europa.eu/doceo/document/TA-9-2020-0359_EN.html .

(44)

    https://eupl.eu/

(45)

   In this context, the ISA2 programme responded by facilitating the exchange of best practices and the re-use of solutions by means of a common repository, the ‘Digital Response to COVID-19’ collection on Joinup.

(46)

   Regulation (EU) 2021/694 of the European Parliament and of the Council of 29 April 2021 establishing the Digital Europe Programme and repealing Decision (EU) 2015/2240 (OJ L 166, 11.5.2021, p. 1).

(47)

   Besides the initiatives mentioned in section 3.3, the following relevant proposals were presented in 2021: the Digital Green Certificate Regulation, a framework for a European Digital Identity, and a Regulation laying down harmonised AI rules.

(48)

   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, 2030 Digital Compass: the European way for the Digital Decade, COM(2021) 118 final of 9.3.2021.