EUROPEAN COMMISSION

Brussels, 20.1.2026

SWD(2026) 2 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

on the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework

{COM(2026) 9 final}

Table of Contents

Glossary

1.Introduction

Purpose and scope of the evaluation    

2.What was the expected outcome of the intervention?

2.1    Description of the intervention and its objectives    

2.2    Point(s) of comparison    

3.How has the situation evolved over the evaluation period?

4.Evaluation findings (analytical part)

4.1.To what extent was the intervention successful and why?

4.2.How did the EU intervention make a difference and to whom?

4.3.Is the intervention still relevant?

5.What are the conclusions and lessons learned?

5.1.Conclusions

5.2.Lessons learned

Annex I: Procedural Information

1. Lead DG, Decide Planning/CWP references    

2. Organisation and timing    

3. Consultation of the RSB    

4. Evidence, sources and quality    

Annex II. Methodology and Analytical models used

1. Overview and process    

2. Data collection and sources    

3. Stakeholder consultation    

4. Workshops and validation activities    

5. Analytical approaches and quality assurance    

6. Limitations and mitigating measures    

7. Points of comparison    

8. Critical assessment of the contractor    

Annex III. Evaluation matrix and, where relevant, details on answers to the evaluation questions (by criterion)

1. Evaluation matrix:    

Effectiveness and impact    

Efficiency    

Coherence    

EU added value    

Relevance    

Annex IV. Overview of benefits and costs [and where relevant, Table on simplification and burden reduction]

Annex V. Stakeholder consultation - Synopsis report

1. Consultation scope and objectives    

2. Mapping of stakeholders    

3. Consultation activities    

4. Call for evidence    

5. Targeted survey    

5.1 Stakeholder participation    

5.2 Summary of results    

6. Targeted consultation (interviews)    

7. SWOT and recommendations workshops    

7.1 SWOT Workshop    

7.2 Recommendations Workshop    

Table of Figures

Figure 1 ENISA’s outputs in policy tasks    

Figure 2 The Cybersecurity Support Action’s support to Member States in preventing and responding to cyber attacks    

Figure 3 Relevance of ENISA’s support to different groups of stakeholders    

Figure 4 ENISA sufficiently exploited synergies with other stakeholders    

Figure 5 Overlaps between ENISA and other stakeholders in the field of cybersecurity    

Figure 6 ENISA’s contribution to promoting cybersecurity cooperation    

Figure 7 ENISA’s contribution to cooperation and coordination between stakeholders    

Figure 8 Achieving ENISA’s objectives without ENISA itself    

Figure 9 Added value of ENISA’s activities    

Figure 10 Objectives that were not reached according to stakeholders    

Figure 11 External factors influencing the ECCF’s objective    

Figure 12 Stakeholders’ contribution to ensuring smooth functioning of the ECCF    

Figure 13 ECCF improvements    

Figure 14 Impact of the CRA proposal on the ECCF    

Figure 15 ECCF added value    

Figure 16 ECCF trust and transparency added value    

Glossary

The table below explains the key terms or acronyms used in this document.

The impact assessment (IA) report that accompanied the proposal for the Cybersecurity Act (CSA) in 2017 1  (ʻ2017 IAʼ) was developed by the European Commission to provide a comprehensive and evidence-based foundation for developing legislation on EU cybersecurity. The main purpose of the 2017 IA was twofold: (i) to assess whether the existing mandate and operations of the European Union Agency for Network and Information Security (ENISA) remained fit for purpose in a rapidly evolving threat and policy landscape, and (ii) to determine the need for an EU-wide cybersecurity certification framework for ICT products and services. The 2017 IA was not only a legal requirement under the then ENISA Regulation 2 but also a response to the growing recognition that cyber threats were increasing in scale and complexity and that fragmented national approaches to cybersecurity and certification risked undermining both the internal market and the EU’s collective resilience.

Purpose and scope of the evaluation

Building on the 2017 IA, this evaluation report examines the performance of ENISA and the European cybersecurity certification framework (ECCF) since the adoption of the CSA. The evaluation was conducted in accordance with Article 114 of the Treaty on the Functioning of the European Union (TFEU), which provides the legal basis for EU action in this area, aiming to harmonise the laws of the Member States to ensure the proper functioning of the internal market. The internal market legal basis for ENISA has been recognised by the Court of Justice (C-217/04, judgment of 2 May 2006) and was further confirmed by the 2013 ENISA Regulation setting out ENISA's current mandate. Activities aimed at increasing cooperation and coordination and EU-level capabilities to complement the action of Member States fall within the field of operational cooperation. This is specifically identified by the Directive (EU) 2016/1148 of the European Parliament and of the Council of 6 July 2016 concerning measures for a high common level of security of network and information systems across the Union (hereafter referred to as NIS Directive), for which Article 114 TFEU is the legal basis. The Directive identifies operational cooperation as an objective to be pursued by the Computer Security Incident Response Team Network (CSIRTs Network), with ENISA providing the secretariat and actively supporting cooperation (Article 12(1)). Article 12(f) further identifies the following as tasks of the CSIRT Network: identifying further forms of operational cooperation, including in relation to categories of risks and incidents, early warnings, mutual assistance and principles and modalities for coordination when Member States respond to cross-border risks and incidents. Article 11(3) under the NIS2 Directive 3 also confirms this.

The 2017 IA covered the performance, governance and organisational structure of ENISA, focusing on the period from 2013 to 2016, also taking into account more recent developments and anticipated future needs. The 2017 IA examined ENISA’s support to Member States, its role in policy development and capacity building, its contribution to operational cooperation and its visibility and added value at both national and EU levels. In parallel, the IA addressed the emerging challenge of cybersecurity certification. It analysed the proliferation of national schemes, the lack of mutual recognition and the resulting risks of market fragmentation and increased compliance costs for businesses, particularly SMEs. The 2017 IA thus considered not only the effectiveness of ENISA’s existing mandate but also the potential benefits and design of an ECCF.

This evaluation assesses the extent to which the objectives of ENISA’s mandate and of the ECCF have been achieved, the effectiveness and efficiency of the measures implemented, their continued relevance, coherence with other EU policies and the added value of EU action. The evaluation addresses the main issues arising from the evolving cybersecurity landscape, such as the increasing number and sophistication of cyberattacks, the need for a high common level of cybersecurity across the EU and the fragmentation of certification schemes for ICT products and services.

In line with the European Commission’s Better Regulation Guidelines, the evaluation applies all five compulsory criteria:

·Relevance: Examining whether the action continues to address the needs of stakeholders in light of technological and threat developments.

·Effectiveness: Assessing the extent to which the objectives of the existing framework have been achieved.

·Efficiency: Evaluating whether the resources invested are justified by the results obtained.

·Coherence: Analysing both internal coherence across the various provisions of the CSA, including ENISA’s mandate and the ECCF, and external coherence in relation to other EU legislation such as Directive (EU) 2016/1148 of the European Parliament and of the Council of 6 July 2016 concerning measures for a high common level of security of network and information systems across the Union, later replaced by Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union (NIS1 & NIS2 Directive), Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (the General Data Protection Regulation or GDPR), Regulation (EU) No 910/2014 of the European Parliament and of the Council of 23 July 2014 on electronic identification and trust services for electronic transactions in the internal market and repealing Directive 1999/93/EC (the eIDAS Regulation), and sectoral rules.

·EU added value: Considering whether action at EU level provides benefits that could not be achieved by Member States acting alone, particularly in terms of reducing fragmentation and supporting the functioning of the internal market.

The methodology for this evaluation combines a review of legal and policy documents, ENISA’s annual reports and technical studies, along with other grey literature, with both quantitative and qualitative data analysis. The evidence base is further strengthened by stakeholder consultation, a call for evidence, targeted surveys, interviews and workshops involving public authorities, EU institutions, industry, academia, civil society and individual citizens. Where possible, the assessment benchmarks EU approaches against international best practices and considers the coherence of measures with other relevant policies.

Despite its broad evidence base, the evaluation has several limitations. Data gaps persist in certain areas, particularly regarding the uptake and impact of certification schemes and the experiences of small and medium enterprises (SMEs) and end-users. The dynamic nature of the cybersecurity landscape and the concurrent evolution of related EU policies make it challenging to decide whether to attribute observed impacts solely to ENISA or to the certification framework. Furthermore, the representativeness of some stakeholder consultations is limited by the relatively low participation of certain groups.

The evaluation covers the period between 2019 and 2023. More recent developments, such as the Cyber Resilience Act (Regulation (EU) 2024/2847), could only partly be considered. The geographical scope includes all EU Member States, as well as countries participating in relevant EU cybersecurity initiatives, such as those in the European Economic Area and the European Free Trade Association and Horizon 2020-associated countries. The methodology and its limitations are described in detail in Annex II to the evaluation report.

This Chapter introduces the EU’s action on cybersecurity as it falls within the scope of this evaluation, outlining its objectives, logic and the baseline conditions that shaped its design. The chapter explains how the CSA and the establishment of ENISA’s mandate and the ECCF were intended to address persistent fragmentation, inconsistent protection and market barriers across Member States, based on the 2017 IA. By reconstructing the state of play prior to the CSA, this chapter provides the necessary context for evaluating the effectiveness and impact of the action in subsequent sections.

2.1Description of the intervention and its objectives

The CSA, adopted in 2019, was designed to address persistent fragmentation and insufficient coordination in the EU’s approach to cybersecurity. The preceding IA identified the need for a more coherent EU-wide approach, as existing legislation such as the NIS Directive, GDPR and eIDAS Regulation had resulted in a patchwork of national policies and certification schemes. This fragmentation undermined both the resilience of the internal market and the competitiveness of European industry.

The preferred options were a ‘reformed ENISA’ (Option 2) and the establishment of an EU general ICT security certification framework (Option 3). The explanatory memorandum confirms that these options were fully taken on board in the final CSA. ENISA was granted a permanent mandate and a central role in the EU cybersecurity ecosystem, with expanded responsibilities to support policy implementation, capacity building, support for operational cooperation and certification. The CSA also established the ECCF, aiming to harmonise certification schemes across the EU, reduce costs and administrative burdens and increase trust in ICT products, processes and services.

These actions were intended to address several interrelated problems. Fragmentation of cybersecurity policies and national certification schemes across Member States led to inconsistent levels of protection and market barriers. There were also dispersed resources and approaches among EU institutions, agencies and bodies, as well as insufficient awareness and information among citizens and businesses regarding cyber threats and the security properties of ICT products and services.

To tackle these problems, the following objectives were agreed upon:

·increase capabilities and preparedness of Member States and businesses,

·improve cooperation and coordination across Member States and EU institutions, agencies and bodies  

·enhance EU-level operational capacity, particularly in the case of cross-border cyber crises

·raise awareness of cybersecurity issues among citizens and businesses,

·increase transparency of cybersecurity assurance for ICT products, services and processes and 

·avoid further fragmentation of certification schemes and related requirements across the EU.

At the time of adoption, the expected achievements were clear. The CSA was expected to deliver a more harmonised and resilient cybersecurity landscape, with ENISA acting as a centre of expertise, supporting policy implementation, capacity building and operational cooperation. The ECCF was expected to streamline certification processes and reduce costs by up to 80% for certain products. The intervention logic was that by empowering ENISA and establishing a harmonised certification framework, the EU would be better equipped to respond to cyber threats, support the digital single market and protect citizens and businesses. Success was expected to be reflected in a more harmonised and resilient cybersecurity landscape, with reduced costs and barriers for businesses and increased trust and awareness among users.

The strategic objective of the intervention logic for the EU cybersecurity certification scheme was formulated as follows: Create a European ICT Security Certification Framework that at the same time, avoids the fragmentation resulting from different approaches across European Union and is as close as possible up to international standards in order to reduce trade hindrances.

Quantitatively, according to the 2017 IA, before action was taken, certification costs for products like smart meters could exceed EUR 300 000 for two markets, with processes taking six to eighteen months. For cloud services, compliance costs were estimated at EUR 1.2 billion, representing up to 10% of annual expenditures and certification could take up to nine months. The CSA aimed to reduce these costs by up to 80% for smart meters and to achieve yearly savings of EUR 1.1 billion in the EU public sector for cloud services, with certification times reduced to four to six months.

2.2Point(s) of comparison 

This section reconstructs the state of play in the EU as of 2017, drawing on the 2017 IA and its supporting studies, stakeholders’ consultations and economic analyses. The baseline scenario reflects the situation before the adoption of the CSA and serves as the main point of comparison for assessing subsequent developments. It covers ENISA’s status and resources, the certification landscape, relevant market and economic data, as well as stakeholder perceptions and consultation data.

ENISA’s status and resources

In 2017, ENISA was operating under a fixed-term mandate that was due to expire in 2020. Its annual EU contribution was EUR 10.3 million and it had an authorised establishment plan of 48 staff members, making it one of the smallest EU agencies in terms of both budget and personnel. Despite its broad mandate, which included support for policy development and implementation, capacity building, community building and support for operational cooperation, ENISA’s resources were widely recognised as insufficient to meet the growing and evolving demands from Member States, EU institutions and the private sector.

When ENISA’s performance was evaluated for the period 2013-2016, it was found to be relevant and efficient to a large extent, but its effectiveness, coherence and EU added value were only partially achieved. The fixed-term mandate was a significant limitation, as it prevented long-term planning and sustainable support for Member States and EU institutions. Furthermore, ENISA’s ability to recruit and retain highly qualified experts was hampered by its location (split between Athens and Heraklion) and the predominance of fixed-term contracts, which made it less attractive compared to other agencies or the private sector.

Stakeholder consultations reinforced these findings. A majority of respondents considered ENISA’s size insufficient for its workload. There was a broad consensus that both its resources and mandate needed to be adapted to enable it to support Member States in facing future cybersecurity challenges. While ENISA’s activities were generally coherent with those of other organisations, there was a clear need for a more coordinated approach at EU level. The agency’s main added value was seen in its ability to enhance cooperation between Member States and communities under the NIS Directive, but its impact was limited by its scale and temporary mandate.

Certification landscape

The certification landscape in the EU in 2017 was highly fragmented and complex. Multiple national and sectoral certification schemes existed. Manufacturers often had to certify the same product multiple times to access different national markets. For example, according to the 2017 IA 4 , smart meter manufacturers faced costs of around EUR 1 million to certify products in three countries, a barrier particularly penalised SMEs.

The SOG-IS Mutual Recognition Agreement (MRA) was the main European mechanism for certification, but it only included 12 Member States plus Norway and was limited to a few protection profiles for certain digital products. Certification processes were lengthy and costly: a CC certificate for the lowest assurance level could take six months and cost EUR 20 000, while higher assurance levels could take up to two years and cost at least EUR 500 000. The lack of a harmonised approach led to significant market fragmentation, increased costs and barriers to entry, especially for smaller companies.

Surveys conducted as part of the 2017 IA showed that a majority of respondents were aware of multiple certification schemes for the same product or service and a large majority agreed that mutual recognition was desirable at European level. The absence of a common EU framework for certification was widely seen as a major obstacle to the development of the digital single market and the competitiveness of European industry.

Relevant market and economic data

According to the 2017 IA, the economic impact of cybercrime in the EU was substantial. It was estimated at 0.41% of GDP, or around EUR 55 billion in 2013, with Germany being the most affected Member State (1.6% of GDP) 5 . The cost of certification for smart meters was at least EUR 300 000 for two markets and for cloud services, compliance costs were estimated at EUR 1.2 billion, representing 2% to 10% of companies’ annual expenditure. Certification processes for cloud services could take 7-9 months.

The EU’s investment in cybersecurity was below the critical mass needed to protect the economy and institutions, especially when compared to international competitors. For example, the US government invested over EUR 19 billion in cybersecurity in 2017, a 35% increase from 2016 6 . EU funding for cybersecurity projects under Horizon 2020 was about EUR 600 million for 2013-2020, with additional contributions from other programmes, but these were not sufficient to address the scale of the challenge 7 . The lack of sufficient investment and coordination at EU level was seen as a key barrier to building resilience and supporting the digital single market.

Stakeholder perceptions and consultations data

Stakeholder consultations for the 2017 IA included a call for evidence, targeted surveys, workshops and interviews with a wide range of actors, including public authorities, EU institutions, industry, academia, civil society and individual citizens. The main findings were as follows:

·A large majority of respondents positively assessed ENISA’s performance for 2013-2016 and a significant proportion considered ENISA to be achieving its objectives.

·Many appreciated ENISA’s products and services as coming from an EU-level body and valued their quality.

·A large majority considered current EU instruments and mechanisms insufficient or only partially adequate to address cybersecurity challenges and almost all saw a need for an EU body to respond, with most identifying ENISA as the right organisation.

·A significant proportion of respondents to the ENISA consultation saw a role for ENISA in establishing a harmonised framework for ICT security certification.

·Many respondents to the 2016 consultation for the preparation of the EU cybersecurity contractual public-private partnership (cPPP) 8 9 , which devoted a section to the topic of certification, did not know whether national certification schemes were mutually recognised, with only a minority saying ‘Yes’ 10 .

·A substantial proportion thought existing certification schemes did not support the needs of Europe’s industry, and many said it was not easy to demonstrate equivalence between standards, certification schemes and labels.

·Many respondents experienced barriers to market access and export due to fragmentation and a significant number of SME respondents identified the cost of certification as a problem.

Stakeholders consistently identified the most urgent gaps to be in cooperation across Member States, capacity to prevent and resolve large-scale attacks and the need for harmonised standards and certification. The consultations also revealed that while ENISA was valued for its expertise and its role in community building, there was a strong call for a more permanent, better-resourced agency with a clearer mandate and greater operational capacity.

In summary, the baseline scenario in 2017 was a small, under-resourced ENISA with a fixed-term mandate, a fragmented and costly certification landscape, significant economic losses due to cybercrime and widespread stakeholder recognition of the need for greater EU-level coordination, harmonisation and investment. The lack of mutual recognition of certification schemes, high costs and long processes and insufficient EU-level operational capacity were seen as major barriers to a secure and competitive digital single market. These baseline conditions provide the foundation for assessing the effectiveness and impact of the CSA and the reformed ENISA. The evidence presented here is drawn directly from the 2017 CSA IA, including its executive summary, problem definition, baseline scenario, economic analysis and stakeholder consultation sections.

This chapter reviews how the EU’s cybersecurity landscape has evolved over the evaluation period, highlighting the transformative impact of major legislative acts such as the Cybersecurity Act, NIS2 Directive, Cyber Resilience Act and Cyber Solidarity Act (both under development at the time of the evaluation). These laws, together with sector-specific regulations, have reshaped the mandates and activities of ENISA and the ECCF. The chapter outlines how new threats, rapid digitalisation and geopolitical tensions have driven regulatory and operational changes, setting the context for assessing the effectiveness, coherence and added value of the EU’s action on cybersecurity.

Current state of play

Since the adoption of the CSA, the EU’s cybersecurity environment has undergone profound changes. The world has seen a dramatic escalation in cyber threats, with attacks becoming more frequent, sophisticated and impactful. Geopolitical tensions, such as Russia’s war of aggression against Ukraine, have brought cyber operations to the forefront of hybrid warfare. At the same time, the COVID-19 pandemic accelerated digitalisation, exposing new vulnerabilities as remote work and digital services expanded rapidly. These developments have fundamentally altered the risk landscape and prompted the EU to significantly strengthen its legal and policy framework for cybersecurity.

Expansion of the EU cybersecurity legal and policy framework

In response to these evolving challenges, the EU adopted new legislative and policy instruments (of which some were under development during the evaluation period), building on the foundation laid by the CSA, and which have been central to the EU’s cybersecurity strategy:

·NIS2 Directive (Directive (EU) 2022/2555): This Directive makes a broader range of entities subject to cybersecurity requirements, strengthens risk management and incident reporting obligations and develops cooperation among Member States.

·Cyber Resilience Act (CRA, Regulation (EU) 2024/2487, under development during the evaluation period): The CRA introduces horizontal cybersecurity requirements for products with digital elements, aiming to ensure that hardware and software placed on the EU market are secure by design and by default considering their lifecycle.

·Cyber Solidarity Act (CSoA, Regulation (EU) 2025/38, under development during the evaluation period): The CSoA focuses on building Union capacities for detection, preparedness and response to significant and large-scale cyber incidents, including the establishment of the EU Cybersecurity Reserve.

·Digital Operational Resilience Act (DORA, Regulation (EU) 2022/2554): DORA sets out comprehensive requirements for the financial sector to ensure operational resilience against ICT-related incidents.

·Network Code on Cybersecurity for Electricity (NCCS, Delegated Regulation (EU) 2024/1366, under development during the evaluation period): The NCCS introduces sector-specific cybersecurity rules for the electricity sector, particularly for cross-border flows.

·EU Regulation on Information Security (PART-IS, Regulation (EU) 2023/203): PART-IS establishes information security requirements for the air transport sector.

·5G Cybersecurity Toolbox (Commission Recommendation (EU) 2019/534): This provides guidance for securing 5G networks across the EU by recommending strategic and technical measures.

·Cybersecurity Skills Academy (COM(2023) 207 final): This initiative addresses the growing gap in cybersecurity skills by promoting training and capacity building.

·EU Action Plan on the Cybersecurity of Hospitals and Healthcare Providers (COM(2025) 10 final): This plan aims to strengthen the resilience of the healthcare sector.

Together with the CSA, these legal and policy instruments form a comprehensive and multi-layered framework that addresses both horizontal and sector-specific cybersecurity challenges. They reflect the EU’s recognition that cybersecurity is not only a technical issue but also a matter of economic security, public safety and strategic autonomy.

Implementation and operational developments

The implementation of the new legal framework has required significant adaptation by Member States, EU institutions and businesses. ENISA’s mandate has expanded accordingly. ENISA now supports Member States in developing and updating national cybersecurity strategies, implementing new legal requirements and building operational capacity. By virtue of other legislation proposed at the time of evaluation and subsequently through the contribution agreements, ENISA has also been tasked with managing the European Vulnerability Database and supporting the EU Cybersecurity Reserve.

The ECCF, established by the CSA, has continued to evolve. ENISA has coordinated the development of candidate certification schemes, such as the EU Common Criteria (EUCC) and the forthcoming European Cloud Certification Scheme (EUCS). However, the process has been slower than anticipated, with the first scheme taking nearly five years from initiation to adoption. The reasons for the delays include the complexity of technical, legal and political negotiations, as well as the need for consensus among a wide range of stakeholders.

Monitoring arrangements have included regular reporting, stakeholder consultations and the use of indicators such as the number of training courses delivered, publications produced, and stakeholder engagement events held. ENISA’s outputs have increased in both volume and scope, reflecting the growing demands on it.

Current situation and key developments

The cybersecurity threat landscape has become more complex and interconnected. Ransomware attacks, supply chain compromises and hybrid threats are now commonplace, affecting critical infrastructure, public services and businesses of all sizes. The average cost of a major cyber incident has risen sharply, with global estimates of overall cost exceeding EUR 5.5 trillion in 2020 11 and projected to reach EUR 9 trillion by 2025.

ENISA has responded by scaling up its activities, delivering a growing number of publications, technical guidelines and capacity-building events. ENISA has played a central role in supporting the implementation of new legislation, advising on incident response and facilitating information sharing among Member States. The ECCF has begun to deliver candidate certification schemes (with one scheme adopted), but the pace has been slower than originally envisaged and fragmentation persists due to differences in national approaches and resource allocation.

Delays, challenges and external factors

The implementation of the new legislative framework has not been without challenges. Delays in the adoption of certification schemes have limited the harmonisation of cybersecurity assurance across the EU. Both ENISA and national authorities are affected by resource constraints, including the shortage of skilled cybersecurity professionals. The regulatory landscape is complex, with overlapping horizontal and sector-specific requirements. This has created compliance challenges for businesses operating across multiple sectors.

External factors, such as the acceleration of digital transformation, the emergence of disruptive technologies like AI and quantum computing and intensifying geopolitical tensions, have all impacted the implementation of the EU’s cybersecurity framework. Hybrid attacks, in which cyber operations are combined with other forms of aggression, have become a defining feature of the threat landscape.

Monitoring, indicators and further information

Throughout the evaluation period, ENISA’s mandate and the ECCF have been monitored using a range of quantitative and qualitative indicators. These include the number of certification schemes adopted, the volume and reach of ENISA’s outputs, the level of stakeholder engagement and feedback from stakeholder consultations. Further details and data can be found in the annexes to the evaluation report.

In summary, the evolution of the EU’s cybersecurity landscape over the evaluation period has been shaped by a series of ambitious and far-reaching legislative initiatives. These new laws have been instrumental in driving change, expanding the regulatory framework and strengthening the capacity of Member States, EU institutions and businesses to address increasingly complex and dynamic cyber threats. Their implementation has also revealed important challenges, including delays, resource constraints and the ongoing need for harmonisation. The rapidly changing cyber threat landscape, technological advances and geopolitical developments increase the critical importance of these legislative measures and the need for continued adaptation and coordination at European level. This is the background to the subsequent analysis of the effectiveness, efficiency and added value of the CSA, ENISA’s mandate and the ECCF.

This chapter presents an analytical assessment of the CSA, ENISA’s mandate and the ECCF, drawing on the study to support the evaluation of ENISA and the ECCF from 2024 12 . This chapter summarises the findings around the five evaluation criteria: effectiveness, efficiency, coherence, EU added value and relevance. Drawing on qualitative and quantitative evidence, including stakeholder surveys, interviews and performance data, this chapter compares the expected outcomes of the action with the actual situation observed during the evaluation period. The analysis provides an evidence-based assessment on the extent to which the action achieved its objectives, the difference it made for various stakeholders and its ongoing suitability in the face of evolving cybersecurity challenges and policy needs. References to supporting data and further details are provided throughout, with additional evidence available in the annexes.

4.1.To what extent was the intervention successful and why? 

Effectiveness

ENISA

ENISA has fulfilled its mandate by delivering nearly all planned outputs. During challenging times, such as the COVID-19 pandemic and Russian war of aggression on Ukraine, ENISA demonstrated flexibility and was evaluated positively by stakeholders. ENISA’s effectiveness stems from a robust governance structure and a matrix-based organisational model that facilitates task delivery and cooperation. However, while stakeholders valued ENISA’s capacity-building efforts, there were occasionally delays due to ongoing resource constraints and rigid strategic plans. In particular, ENISA planned 219 outputs between 2017 and 2022 and delivered 203, often exceeding initial targets per output due to a higher-than-expected number of stakeholders engaged in its activities. Eight outputs were delivered to some extent and in three cases, outputs were not delivered or only partially achieved, due particularly to the COVID-19 pandemic. The results of five outputs were not reported entirely. ENISA’s support for policy implementation was generally well received, with 93% of stakeholders expressing satisfaction with ENISA’s added value in this area in 2022. ENISA contributed to national and EU policies and legislative initiatives, supported the implementation of the NIS Directive and played a key role in the development of the ECCF. In 2021, ENISA made 193 relevant contributions to EU and national policies and legislative initiatives, which increased to 314 in 2022. However, over 80% of these contributions were through organising workshops and conferences. In 2022, ENISA’s reports, analyses and studies were referred to 65 times at EU and national levels. Between 2017 and 2023, ENISA produced and issued a total of 286 publications. These covered a wide range of topics, with the most frequent being cybersecurity policy (72 publications), cyber threats (48), critical infrastructure (38), incident reporting (24) and emerging technologies (21).

Among key outputs, ENISA’s studies, such as the 2020 report on NIS investments 13 , highlighted the challenges faced by organisations in implementing the NIS Directive, such as unclear expectations and limited support from national authorities. Moreover, the Agency’s support to the NIS Cooperation Group Workstream on Digital infrastructure and service providers was crucial in analysing the security aspects of the digital infrastructure sector, which then fed into the NIS2 proposal to extend the scope of the sector. This analysis also assisted Member States in the development of guidelines by highlighting priorities and best practices within the industry. 14  

In 2023, ENISA’s efforts in the implementation of the directive included the provision of technical advice to the Commission on the implementation of NIS2 security measures and the drafting of guidelines for national coordinated vulnerability disclosure policies. 15 The Agency supported the implementation of NIS2 by responding to 12 individual requests from Member States for advice on the transposition of the Directive into national legislation and by organising risk management training for national authorities to help build knowledge and expertise.

However, stakeholders also noted that ENISA’s reports could be more concise, practical and better tailored to their needs, with improved visualisation and summaries. The lack of a more effective communication system and the complex structure of ENISA’s website further complicated stakeholder interaction with ENISA’s outputs. Some stakeholders called for more transparent processes, better tailored content and more detailed and pragmatic guidance, especially for sector-specific needs.

While ENISA has achieved many of its objectives, there is a clear opportunity for ENISA to enhance its efficiency through improved prioritisation, clear focus and more strategic resource allocation. ENISA’s efficiency has been occasionally hampered by external factors and a need for more streamlined internal governance. Internally, ENISA has developed innovative solutions, such as a matrix-based organisational model, to coordinate between operational and administrative functions more effectively. However, ENISA's staff occupancy rate fluctuated between 74% and 90% from 2017 to 2023 and recruitment and retention of senior cybersecurity experts remained a persistent challenge. This impacted deliverables, with some outputs that were not delivered or only partially achieved.

There is also room to improve task prioritisation. By reassessing its operational focus, ENISA can leverage existing frameworks and stakeholder feedback to improve task delivery even further. This approach is particularly crucial for addressing emergent priorities without compromising on existing commitments. ENISA’s ongoing commitment to stakeholder engagement and consultation has been a strength, yet its operational capacity shows insufficient resources. More dynamic reallocation of tasks and resources could help towards timely fulfilment of new requests and enhance ENISA’s ability to respond to cybersecurity challenges to a certain extent.

ENISA’s aim of maintaining and bolstering its reputation in the cybersecurity community could be further realised by ensuring that tasks are aligned not only with strategic goals but also with operational capacity. To this end, ENISA’s mandate would need to be revised, supported by a continual process of resource evaluation and strategic reprioritisation. This structured approach will assist ENISA in navigating its evolving role within the EU cybersecurity framework more effectively, ultimately increasing its impact and sustaining its status as a flagship institution for cybersecurity initiatives. Such prioritisation should be facilitated through collaborative efforts between ENISA, Member States, and EU policymakers to ensure alignment with strategic objectives and operational capacity.  

ECCF

The ECCF was envisioned as a pillar for improving cybersecurity assurance across the EU internal market, aiming to harmonise the certification of ICT products, services, and processes. It sought to tackle persistent issues such as market fragmentation, the need for enhanced transparency, and raise public trust in digital solutions. Its structured governance model involving entities like ENISA and the European Cybersecurity Certification Group (ECCG), gathering National Cybersecurity Certification Authorities (NCCAs), has indeed laid a foundation for increased coordination among stakeholders, including Member States and private entities. However, the practical realisation of these goals has been met with numerous challenges that have restricted the ECCF’s effectiveness. A significant shortcoming of the current ECCF is its inability to effectively address the fragmentation of certification schemes across the EU, mainly due to procedural limitations. This fragmentation has persisted even though the framework is intended to harmonise certification processes, leading to inconsistencies and inefficiencies in cybersecurity assurance. This can be observed in the substantial delay in operationalising the first certification scheme, the EUCC, which took 55 months from initiation to adoption. This delay points to inefficiencies, predominantly influenced by the complex and multifaceted approval procedures. Moreover, there is a certain lack of clarity about responsibilities and accountability among stakeholders. This has made achieving the framework objectives even more complex. External factors further complicated the ECCF’s aims.

The evolving geopolitical landscape, characterised by increasing cyber threats and political tensions around data sovereignty and digital control, required adaptive measures that the ECCF struggled to implement swiftly. These external pressures resulted in delayed scheme adoptions, as seen with the EUCS, where discussions stalled over non-technical debates like data localisation requirements.

Despite these hurdles, there have been positive outcomes–particularly in raising awareness across Member States about the importance and intricacies of cybersecurity certification. The COVID-19 pandemic, while causing operational delays, also made the necessity for resilient digital infrastructure even clearer, thrusting cybersecurity into the policy spotlight. Additionally, the analysis identified key lessons, noting the uneven resource allocation across stakeholders, which hinders the uniform development and deployment of certification schemes. National authorities acknowledged the support of the ECCF in building national cybersecurity certification capabilities but noted significant resource imbalances between Member States. For example, the number of FTEs was seven times higher in one Member State compared to another one for similar activities of issuing certificates. Also, ENISA stated that due to the difficulty of retaining staff and the competitive job market for cybersecurity professionals, it could not maintain its full staff level (including for certification) at times. Addressing these disparities is vital for future efficiency and effectiveness, particularly through retaining expert staff within ENISA and fostering constant dialogue among all parties involved.   

Despite these challenges, the ECCF did improve harmonisation among Member States and established better cooperation opportunities, particularly through the creation of stakeholder cooperation forums such as the ECCG. Most participants during the interview process agreed that the ECCF established better cooperation opportunities for Member States and the majority of Member State representatives interviewed agreed that the ECCF improved harmonisation among Member States. Statements gathered during interviews underlined the importance of the ECCF support to Member States when Member States developed their capabilities. For instance, one national authority reported how it benefited from cooperation among Member States by acquiring knowledge in areas where it lacked expertise, while others stressed the pedagogical value that cooperation among ECCG members can have. However, national experts stressed that the ECCF strengths remained ‘potential’ due to the lack of scheme implementation so far.

Efficiency

ENISA

During the evaluation period, which spanned from 2017 to 2023, ENISA demonstrated efficient operations under its existing governance structure. The matrix-based organisational framework helped ENISA prioritise tasks, optimised resource alignment, and fostered cooperation between various units. This approach, coupled with a balanced mix of operational and administrative staff, helped facilitate the execution of its mandated duties. Despite this, the evaluation highlighted several key areas where ENISA has room to improve its efficiency. Interviews with ENISA’s staff, stakeholder surveys and internal documentation indicated that ENISA struggled to keep pace with increasing demands and fill specialised positions, exacerbated by a global shortage of IT and cybersecurity specialists. This has led to delays, reprioritisation of tasks, and periods of high stress and workload. Stakeholders gave a positive assessment of ENISA’s performance during periods of high workload, with 63% of respondents stating that ENISA was successful or very successful in delivering its outputs. However, resource constraints, operational inefficiencies and challenges stemming from the political and regulatory environment were identified as the main obstacles to ENISA’s performance during periods of high workload. In 2022, 64% of respondents experienced stress due to high workload and 43% due to high administrative burden. Only 32% gave a positive assessment of stress management.

Nevertheless, certain adjustments could alleviate these challenges. The recent strategic decisions to reallocate human resources demonstrate a capacity that can address priority shifts to a certain extent. For instance, the reallocation of approximately 10.5 FTEs planned in 2022 to accommodate the Cybersecurity Support Action suggests that ENISA is able to optimise current resources when necessary even though, on that occasion, the FTEs needed for the implementation of the Support Action were in fact obtained partially through procurement and partially through contract management staff.

Budget management also presents opportunities for improvement. Despite significant budget growth from 2017 to 2023, resource constraints persisted. ENISA’s budget grew unevenly, with notable increases in 2019 (over 46%), 2020 (30.5%) and 2022 (72.4% compared to 2021, due to the Cybersecurity Support Action). ENISA was less able to balance approved and committed appropriations between 2019 and 2022 due to delays in actions like the Cybersecurity Support Action. By reversing this trend and making efforts to manage administrative expenditure, including addressing procurement delays, internal efficiency could see certain further enhancement. 

ECCF

The efficiency of the ECCF has been subject to scrutiny, given the extended timelines for the adoption of cybersecurity certification schemes and the myriad of complexities involved. Despite its strategic aim of streamlining the certification process across the EU, the ECCF’s efficiency was notably hampered by drawn-out discussions and preparation phases that culminated in significant delays; the first scheme EUCC was only adopted in early 2024, nearly five years post-implementation. These protracted timelines can be attributed to multifaceted challenges encompassing both political and technical dimensions.

Given the recent adoption of the EUCC scheme, it is premature to identify costs borne or benefits experienced by stakeholders for compliance with ECCF requirements. Survey respondents indicated some costs and benefits related to preparatory activities. Benefits include enhanced cooperation, knowledge exchange, growing awareness and contribution to standard development. Costs include dissemination and awareness support, pilot implementations, legal consultations for aligning national requirements, setting up of web portals and reporting mechanisms and CAB accreditation. Stakeholders involved in preparing schemes bore costs related to scheme development, publication and communication efforts, including investment in staff allocation and upskilling.

Content and process-related issues had the greatest impact on the efficiency of the ECCF. Content issues included political factors and the technical complexity of schemes, which varied for each scheme depending on the stakeholders involved and products/services to be certified.

More generally, political challenges, including the politicisation of discussions around certification requirements, have hindered progress by creating an environment where transparency and communication suffered. For instance, the EUCS was impacted significantly by debates around data sovereignty requirements, attracting political pressure from non-EU countries and industry outside the EU, leading to shifts from technical to political discourse within the ECCG. Moreover, the scheme was impacted by a shift in EU policy priorities due to the concurrent proposal of the CRA in September 2022.

Technical complexities further contributed to inefficiencies, notably the difficulty in translating draft schemes into legal acts, given the diverse and demanding nature of the products/services slated for certification, such as 5G and cloud computing. The wide-ranging requests and lack of established standards in certain areas added layers of difficulty to the preparation and adoption processes, necessitating a multitude of stakeholders and phases to ensure alignment with existing policies and practices. Stakeholders confirmed the difficulty of converting an existing international certification mechanism (SOG-IS) into EU law and highlighted the need for more structured engagement in the framework through clear processes and realistic timelines. Despite these inefficiencies, several positive elements arose within the framework. The formation of dedicated groups and forums, including the ECCG, the Ad Hoc Working Group (AHWG) dedicated to specific schemes and the Stakeholder Cybersecurity Certification Group (SCCG), facilitated necessary stakeholder involvement. Nonetheless, there remains substantial room for improvement in ensuring these structures function optimally as, for example perceived lack of involvement of the SCCG members in the ECCF. Refinement of internal governance is crucial to increase the active participation and strategic input of stakeholders.  

Coherence

ENISA

In assessing ENISA’s coherence, the evaluation highlights both strengths and areas for improvement. ENISA’s commitment to fostering cybersecurity cooperation at the EU level is apparent, particularly through its facilitation and direct engagement with stakeholders. ENISA supported EU networks such as the CSIRTs network and EU-CyCLONe and facilitated the exchange of best practices through the NIS Cooperation Group and organised exercises such as CyberEurope. Despite some overlaps with national cybersecurity authorities and CERT-EU, ENISA’s efforts were largely complementary, with effective exploitation of synergies and knowledge sharing. Survey data indicated that 74% of stakeholders overall agreed that ENISA sufficiently exploited synergies in expertise and knowledge sharing with other actors, with representatives of private bodies slightly less satisfied (65%).

This dual approach has enabled ENISA to significantly contribute to the cyber domain, aligning with recent legislative frameworks. However, while ENISA’s role as a facilitator and coordinator is positive, several areas require improvement to enhance coherence. The evaluation identified the need to improve synergies between the responsibilities and actions of ENISA and other EU bodies such as the European Cybersecurity Competence Centre (ECCC), as well as national cybersecurity authorities. Although these roles are often complementary, opportunities exist to further streamline operations and improve organisational efficiency. By formalising cooperation arrangements with other entities, such as EMSA and the JRC, ENISA could better leverage synergies and ensure a unified approach to cybersecurity initiatives. Internal communication and resource management within ENISA should also be refined. ENISA’s interaction with private stakeholders and international partners must be more predictable and transparent to maintain confidence and foster collaborative efforts. In this context, private entities, while considering ENISA’s contribution beneficial, suggested that ENISA’s stakeholder engagement activities could be improved, particularly in relation to collaboration with industry representatives and non-EU countries.

In alignment with the CRA and NIS2 Directive, a clear delineation of ENISA’s tasks supporting policy implementation could increase efficiency and ensure consistency across regulatory measures.  This clarity would also improve ENISA’s ability to respond to sectoral regulatory requirements.

In summary, while ENISA has demonstrated a solid foundation in promoting cybersecurity coherence in the EU, there is potential for it to reprioritise its efforts. This approach will help it to efficiently fulfil its mandate and adapt to the evolving cybersecurity landscape. By addressing current inefficiencies and enhancing inter-agency coordination, ENISA can effectively maintain its crucial role within the EU’s cybersecurity framework.  

ECCF

The ECCF’s coherence is affected by the lack of clear accountability mechanisms, which has led to difficulties in aligning its objectives with other legislative measures. This misalignment risks creating overlaps and inefficiencies in the cybersecurity landscape. To ensure a unified approach to cybersecurity, the ECCF needs to be completely coherent with other EU legislative instruments, including the NIS2 Directive and the CRA. In theory, the ECCF is aligned with these legislative measures, designed to address various facets of cybersecurity within the EU landscape, yet real-world integration remains complex and requires diligent oversight. The forthcoming implementation of the EUCC scheme poses a significant test for this coherence: if the scheme is successfully deployed, it will demonstrate the ECCF’s ability to harmonise and effectively leverage additional legislative efforts. Stakeholders have emphasised the need for careful coordination between the ECCF and emerging regulatory acts to prevent potential overlaps which could undermine efficiency and dilute intended effects across sectors. Specifically, concerns arise regarding the interface between the ECCF and the CRA, as both initiatives aim to raise cybersecurity standards but risk redundancy if not fully synchronised. On the sectoral side, coherence must extend to accommodate continuing advancements in technologies, ensuring that cybersecurity initiatives are appropriately nuanced to address critical infrastructure needs.  

Survey data showed that 83% of stakeholders found the ECCF to be coherent with other EU instruments, with 55% rating it as fairly coherent, 23% as very coherent and 5% as perfectly coherent. However, concerns remain about potential overlaps, particularly with the CRA, which could result in duplication of efforts and inconsistent requirements. More than half of respondents identified overlaps between the ECCF and other EU initiatives. Member States stressed the importance of ensuring coherent implementation of the CSA and CRA and highlighted the need to establish communication channels with international organisations to leverage existing European or international standards and prevent inconsistencies between standards developed at the EU and international levels.

4.2.How did the EU intervention make a difference and to whom?

EU added value

ENISA

ENISA has significantly contributed to enhancing the EU’s cybersecurity ecosystem, yet there are opportunities for improvement that could amplify its impact. Serving as a centralised hub, ENISA has facilitated vital cooperation across the EU. It has complemented national efforts, especially in Member States with less developed cybersecurity infrastructures, and aligned cybersecurity practices and policies. Around two thirds of surveyed stakeholders considered that without ENISA, the collection and dissemination of relevant cybersecurity information, the generation of new knowledge, insights and evidence on cybersecurity issues and supporting the implementation of EU cybersecurity policies at the national level would be hard to achieve. Three quarters of respondents believed that without ENISA there would be little effect on improving Member States’ cybersecurity capacities, as well as raising awareness of cybersecurity issues. As a decentralised EU Agency, ENISA’s specialised mandate has allowed it to consolidate cybersecurity expertise and engage effectively with Member States, playing a pivotal role in shaping Europe’s cybersecurity landscape. In this context, ENISA’s main focus on Member States is essential, given its role in providing insights into emerging threats and recommending tools and strategies for addressing them.

Moreover, ENISA plays a critical part in promoting cybersecurity certification and supporting standardisation activities, which helps reduce market fragmentation and fosters robust cybersecurity practices across the EU. Although no other similar bodies with ENISA’s expertise and organisational agility exist, its current primary focus on national authorities has attracted criticism from private sector stakeholders. Feedback from large industry players indicates that more could be done to tailor insights to the private sector’s specific challenges. Companies reported that they often relied on ISO/IEC standards rather than ENISA schemes. Some organisations valued ENISA’s technical guidelines, tools and reports, although they claimed some of them were redundant with existing standards. Though the primary focus on national authorities is crucial, ENISA could address these concerns by strategically improving engagement with stakeholders and collaboration with the industry. Additionally, ENISA’s mandate could benefit from strategically reassessing its priorities to adeptly adapt to evolving cybersecurity challenges. This would allow ENISA to maintain its valuable contributions to the EU, while effectively addressing the expanding needs of its varied stakeholders. 

Key achievements and challenges 

ENISA is widely recognised within the EU’s cybersecurity community for its robust reputation, quality publications and significant role in fostering cooperation among Member States and other cybersecurity entities. ENISA’s work on harmonising cybersecurity requirements is crucial in establishing a consistent level of protection across Member States, contributing directly to capacity building, especially for smaller Member States. This harmonisation not only ensures a secure digital environment across the EU but also elevates cybersecurity preparedness across its stakeholders. 

However, the evaluation identified several challenges faced by ENISA. ENISA shows limited agility in responding to evolving cybersecurity threats which may lead to potential delays in its activities. To mitigate the problems of limited resources, the stakeholders2 emphasised the need for improved recruitment processes and workload management strategies. Expanding ENISA’s mandate to extend its operational role could address these concerns, allowing it to leverage technological advancements and improve cybersecurity frameworks. This restructuring would enable ENISA to proactively tackle dynamic threats, increasing its impact through joint training initiatives and making a contribution to policy-making processes. 

Finally, ENISA’s stakeholder consultation and management systems are deemed effective in facilitating management of stakeholder needs and expectations. However, a stronger, more transparent relationship with Member States is necessary to develop cooperation and information sharing. Future priorities include updating internal frameworks to better manage growing responsibilities and diverse challenges, ensuring that ENISA can fully implement its mandated tasks given its staff size.  

ECCF

Despite its potential, the ECCF has struggled to deliver added value in fostering a unified and effective cybersecurity environment across the EU. The ECCF sought to significantly enhance the EU’s cybersecurity landscape by introducing an unprecedented development procedure and governance structure for certification processes. At its core, the ECCF represented a critical advancement in the EU’s ability to create a harmonised approach to the certification of ICT products, services, and processes. The inherent EU added value of this framework lies in its potential to bridge disparate national approaches, fostering an internal market with consistent, reliable, and recognised cybersecurity standards across Member States. However, the protracted timelines and fragmented implementation have impeded the ECCF’s ability to fully capitalise on its envisioned value. Specifically, the delay in actionable schemes, evidenced by the late adoption of initiatives like the EUCC, curtailed immediate impacts, leaving its theoretical potential largely unfulfilled.

The EUCC was adopted in January 2024, 4.5 years after the Commission request was introduced. Based on interviews with Commission, Member States and ENISA, this long timeframe of development and adoption can be attributed to a mix of factors. First, delay was caused by the lack of experience in developing and adopting schemes. Stakeholders confirmed the difficulty of translating the ENISA candidate scheme endorsed by the ECCG into an EU legal text both for the Commission draft implementing act and during the comitology procedure. Stakeholders also confirmed the difficulty of converting an existing intergovernmental certification mechanism (Senior Officials Group Information Systems Security – SOG-IS) into EU law that raised challenges related both to technical complexities and harmonisation with the repeal of national schemes. Moreover, the scheme was impacted by a shift in EU policy priorities due to the concurrent proposal of the CRA in September 2022, as well as by staff turnover and shortage from the Commission side.

The request for EUCS was introduced in November 2019 building on previous stakeholder-driven efforts that started in 2017. According to surveyed stakeholders, the adoption of the EUCS scheme was primarily impeded by issues related to its content and largely due to the politicisation of the debate, which contributed to divide Member States positions within the ECCG. As emerged from the call for evidence and confirmed by interviews with all relevant stakeholder categories (i.e. national authorities, EU institutions and agencies and industry), the discussions surrounding sovereignty concerns resulted in polarized discussion around the draft scheme in the institutional and public spheres. Furthermore, the situation around the EUCS triggered debates related to the scope of the ECCF as well as criticism from stakeholders regarding insufficient involvement in scheme development processes, due to a lack of transparency.

Disparities in systemic implementation, compounded by varying degrees of readiness and resource availability among Member States, further dilute the framework’s comprehensive influence. Nevertheless, the ECCF has invigorated cooperative dynamics across the EU. By establishing groups such as the ECCG, it has institutionalised coordination efforts, enabling broader engagement across different governance levels. This collaborative infrastructure encourages information sharing and joint strategies, promoting a unified cybersecurity stance against evolving threats. While the direct influence of the ECCF is limited, stakeholders broadly agreed that the framework brings EU added value compared to what could have been achieved by Member States alone. 92% of stakeholders believed that Member States alone could not have achieved more streamlined and cost-effective certification processes, specifically in terms of uptake of certification, cost-effective processes, cyber-awareness, trust in the EU single market and cybersecurity by default and design. Most stakeholders recognise the added value of the ECCF in achieving a more secure, transparent and cohesive internal market for ICT products, services and processes. According to 88% of stakeholders, Member States alone might not be able (or able only in a limited way) to increase trust and awareness of citizens and businesses regarding the cybersecurity of ICT products, processes and services alone. This suggests the ECCF helps enhance trust.

For the ECCF to unlock its full EU added value, increased and concise stakeholder participation is critical. Fostering an inclusive environment where industry partners, national authorities, and EU bodies actively contribute to and guide the certification process will ensure widespread acceptance and effectiveness of cybersecurity standards. 

Key achievements and challenges 

The ECCF serves as a critical tool for enhancing EU-level collaboration between ENISA, Member States, and industries. Therefore, it has considerable relevance in the dynamic cybersecurity environment. Its adaptability facilitates scheme development through ad hoc working groups and aligns with EU legislative frameworks like NIS2. This potential, while significant, remains largely untapped due to limited implementation of actual schemes. 

The ECCF is pivotal in addressing emerging cybersecurity threats and fostering compliance, particularly in leveraging new technologies such as artificial intelligence. Stakeholders acknowledge its role in strengthening cooperation among Member States and improving cybersecurity preparedness. Its value in fostering internal market exchanges, by replacing national certification schemes with EU-wide ones, is also recognised. 

However, the ECCF is hampered by significant weaknesses. Lengthy processes for adopting schemes hinder its effectiveness, a challenge exacerbated by technical complexities and political pressures from industry lobbying. These delays undermine trust and prevent the swift adoption of standards. Opportunities exist for the ECCF to enhance EU cybersecurity frameworks, yet threats like resource constraints and geopolitical tensions pose challenges. Shifts in policy priorities and potential legislative overlaps could make the framework less effective, risking market inconsistencies. Addressing these issues is vital for ensuring the ECCF’s evolution from a promising concept into a fully operational mechanism, bolstering EU cybersecurity standardisation and certification.  

4.3.Is the intervention still relevant?

Relevance

ENISA

ENISA’s relevance within the cybersecurity domain is also due to its responsiveness to evolving stakeholder needs and its flexibility to adapt to the changing landscape. ENISA has consistently demonstrated its ability to review and realign its areas of action to address emerging developments, thereby maintaining its position as a vital component in the EU’s cybersecurity framework. For example, in 2020, ENISA established the Ad hoc Working Group on Artificial Intelligence Cybersecurity to address the growing need to map the AI threat landscape and develop security measures. In 2021, it set up an interdisciplinary working group on emerging and future cybersecurity challenges, integrating foresight into cybersecurity practices and increasing awareness of future threats. 

While stakeholder satisfaction with ENISA’s efforts is generally positive, 44% of the surveyed stakeholders and 50% of industry representatives indicated their needs were met only ‘somewhat’, ‘to a small extent’, or ‘not at all’ by ENISA’s services and outputs. There are dimensions where its relevance can be further enhanced. Despite its responsive nature, ENISA could still further improve support and increase visibility among diverse sectors and stakeholders, particularly for SMEs, which often struggle to adhere to cybersecurity requirements. A shift towards providing more direct tools and resources tailored to specific sectors but also providing insights in and tools to address emerging threats can increase ENISA’s impact. ENISA’s approach to stakeholder engagement was effective, utilising forums, committees and working groups to actively involve national experts in operations and publications. However, the complex decentralised structure within some Member States posed challenges that could be mitigated by better organisation and clearer coordination with national authorities. ENISA’s ongoing initiatives, including the development of cybersecurity guidelines and capacity-building programmes, reflect its commitment to fostering cooperation and reinforcing the EU’s collective cybersecurity posture. Stronger collaboration across industries and improved information access could address some limitations perceived by the industry sector.

What would really help ENISA to vastly improve would be re-evaluating priorities, streamlining processes, acquiring new appropriate resources and maximising existing resources efficiently, thereby reinforcing its foundational role in Europe’s cybersecurity ecosystem. Through strategic alignment with the European cybersecurity strategy, ENISA’s new priorities could create pathways towards contributions with greater impact. For ENISA to improve its capacity to provide policy and technical support, it might need to provide more resources, be more selective with its engagements and refine its operational focus areas. In conclusion, while ENISA’s relevance is clear, there is still room for improvement.   

ECCF

The ECCF has emerged as a crucial response to the growing complexity and sophistication of cyber threats across the EU. It aspires to establish harmonised cybersecurity certification schemes that ensure trust and foster a secure digital market. Despite its promising premise, the framework’s relevance is still considered more potential than practical, with certification schemes only recently entering the operational phase. The lack of tangible results so far is a sign that implementation is uneven and means the ECCF’s current standing in the cybersecurity landscape is uncertain. The significance of the ECCF lies in its strategic role in raising cybersecurity standards and enabling mutual recognition of certifications across Member States, thereby reducing individual enterprise costs and strengthening the internal market. The ECCF seeks robust integration with other EU legal acts, aiming to streamline procedures and facilitate cross-border trade. Yet the protracted timeline for scheme deployment and the discrepancy in expertise and resources among Member States make it much harder for the ECCF to realise its full potential. These challenges hinder collaborative efforts and impede the standardisation process that is central to the ECCF’s mission.  

Despite these challenges, there are several factors ensuring the ECCF’s relevance. One is that the surge in cyber threats intensifies the need for a united cybersecurity strategy that can adapt swiftly to changing scenarios, such as the increased relevance of certification in high-assurance areas like cloud services and 5G infrastructures. Public procurement mandates in these sectors reflect the growing demand for a unified and reliable certification framework that the ECCF can fulfil. Furthermore, the ECCF is linked to emerging legislative acts, notably the CRA and NIS2 Directive, which points to its value in addressing critical infrastructure needs and legal conformity across the EU. ENISA’s proactive role and the establishment of NCCAs mark crucial milestones towards strengthening collaborative interactions and promoting certification adoption. However, there are considerable differences between larger and smaller Member States when it comes to resource allocation and expertise, meaning continued imbalances in participation and effectiveness that impact the development of a collective scheme.  

In conclusion, the evaluation finds that EU action has delivered relevant benefits in strengthening the cybersecurity landscape across the EU. ENISA has established itself as a centre of expertise, a facilitator of cooperation and a key contributor to policy development and capacity building, particularly for Member States with less developed cybersecurity capabilities. The ECCF, while still in the early stages of implementation, has laid the groundwork for harmonised certification and increased trust in digital products and services. However, both ENISA and ECCF have faced significant challenges, including resource constraints, delays in scheme adoption and the complexity of aligning with a rapidly evolving legislative and technological environment. Despite these obstacles, they remain highly relevant and continue to provide clear EU added value, especially in fostering cross-border cooperation, supporting SMEs and addressing emerging threats. The findings highlight the need for ongoing adaptation, increased resources and enhanced stakeholder engagement to ensure that the EU’s cybersecurity framework remains effective, efficient and fit for future challenges.

This chapter synthesises the main conclusions and lessons learned from the evaluation of EU action on cybersecurity, focusing on ENISA and the ECCF. It provides policy-relevant insights for future development while staying within the limits of a staff working document. The conclusions are based on a systematic screening of evidence from performance data, stakeholder consultations and targeted workshops, highlighting what has worked, what remains uncertain and where challenges persist. Lessons learned are presented as part of the narrative, with attention to regulatory simplification and burden reduction.

5.1.Conclusions

The evaluation of ENISA highlights its crucial role in working towards a cohesive cybersecurity landscape across the EU. Although ENISA has shown effectiveness in parts of its mandate and generating valuable outputs, there remains significant room for improvement in handling external disruptions and meeting stakeholder expectations consistently. As demands on ENISA continue to grow, it is important to reassess its mandate and resources to better align it with priorities, with continued emphasis on supporting Member States as they address cybersecurity threats and improve their national cybersecurity infrastructures. Making ENISA more effective would require refinement of its report production process, making reports more tailored to stakeholder needs as well as more user-friendly and accessible through visual aids and concise summaries. Strengthening communication channels is essential to ensure ENISA’s activities and services are clearly visible to stakeholders, including industry players. A well-defined communication strategy will aid in fostering stronger connections and cooperation within existing cybersecurity networks such as ISACs. 

ENISA can become more efficient through a more strategic focus on task prioritisation, enabling a more streamlined approach to managing workload pressures. To become more relevant to stakeholders, ENISA should continue to expand its central role in supporting Member States by strengthening its capacity to provide timely insights into emerging threats and strategic tools for addressing them. Moreover, as a number of stakeholders noted, ENISA could establish more structured and transparent methods of engaging with private entities, with an emphasis on supporting SMEs. Clarity should be sought regarding ENISA’s role in policy implementation with other EU institutions, ensuring that collaboration with Member States is at the forefront of these efforts to reinforce the cohesion of the EU’s unified cybersecurity strategy, which would also include strengthening cooperation with other EU agencies and seeking synergies with other cybersecurity bodies for joint actions to improve operational coherence across Europe. 

Overall, maintaining ENISA’s status as a specialised agency within the EU framework is important, as it ensures continued focus on cybersecurity priorities.

The ECCF has partially achieved its objectives. It has succeeded in improving cooperation and coordination among Member States, EU institutions and the private sector, notably through the creation of forums such as the ECCG and to a lesser extent, the SCCG. These structures have facilitated knowledge sharing and harmonisation, but delays in the adoption of certification schemes, most notably the EUCC, which took nearly five years to implement, have limited ECCF’s impact on market fragmentation, trust and transparency. The lack of implemented schemes means that many of ECCF’s strengths remain ‘potential’ rather than realised and the benefits for businesses and citizens are yet to materialise.

The evaluation finds that the EU’s role has been essential in bringing together diverse actors to work on shared solutions that would not have been possible at national level. ENISA’s decentralised structure and the ECCF’s harmonised approach have enabled economies of scale and safeguarded key EU interests in cybersecurity. Stakeholders consistently noted that, without EU involvement, coordination, expertise development and cross-border cooperation would be significantly weaker, leading to fragmented approaches and increased costs.

Unintended effects have also emerged. The politicisation of debates around certification schemes (e.g. cloud sovereignty requirements) has led to delays and decreased trust among stakeholders. Nevertheless, the overall impact of the action remains positive, with stakeholders recognising the need for continued adaptation and coordination.

In terms of regulatory simplification and burden reduction (REFIT), the evaluation highlights the importance of streamlining reporting requirements, improving communication channels and making ENISA’s outputs more accessible and tailored to stakeholder needs. Stakeholders suggested that more concise, practical reports and greater use of visualisations would enhance effectiveness and reduce unnecessary complexity.

5.2.Lessons learned

The evaluation yields several lessons that are relevant for future policy development, while acknowledging that some findings remain preliminary and require ongoing monitoring.

First, ENISA’s expanding mandate and the evolving cybersecurity landscape would benefit from a flexible approach to resource allocation and prioritisation. ENISA’s effectiveness depends on its ability to match its growing responsibilities with adequate human and financial resources. Lessons from stakeholder consultations indicate that secondment of national officials, partnerships with academic institutions and targeted recruitment programmes could go some way to addressing resource gaps and improving capacity.

Second, the quality of stakeholder engagement is a critical determinant of success. ENISA’s effectiveness, efficiency and relevance are closely linked to its relationships with Member States, EU institutions and industry. The evaluation suggests that ENISA could further develop its cooperation strategies, including clearer communication campaigns, more structured engagement with industry (especially SMEs) and improved feedback mechanisms.

Third, the ECCF’s experience highlights the importance of having clear roles and responsibilities in scheme implementation and maintenance. Despite ENISA’s pivotal role in fostering cooperation and operational cohesiveness among Member States and other stakeholders, constraints on the efficiency and effectiveness of the ECCF have been evident, mainly due to the complexities of scheme adoption processes. These issues highlight the necessity of a substantial revision in governance structures to improve operational clarity and accountability at all levels. The evaluation also points to the importance for more granular scheme requests, early preparatory analysis and regular updates to work programmes to align expectations and smoothen adoption processes.

Fourth, developing streamlined decision-making processes within the ECCF which clarify roles and responsibilities will promote transparency and efficiency, particularly in collaborative efforts among Member States, the Commission, and ENISA. This will foster accountability and reduce inefficiencies.

Fifth, a commitment to setting and adhering to realistic timelines for the development and implementation of certification schemes is essential. This involves supporting detailed technical analysis and bolstering preparatory efforts to effectively anticipate and mitigate political influences. 

Sixth, actively investing in the training and retention of skilled personnel within ENISA is crucial to ensuring continuity and expertise when navigating complex cybersecurity challenges. Long-term workforce stability will be vital to maintaining the ECCF’s operational efficacy.

Seventh, industry and consumer awareness about the ECCF could be increased through targeted campaigns and strategic involvement, emphasising the value of certified products and services. A proactive approach to garnering stakeholder support is critical to boosting demand and trust in ECCF initiatives.  

Eighth, the relevance and added value of these efforts remain ‘potential’ until schemes are fully implemented and maintained. Promoting uptake among industry, raising awareness among consumers and linking scheme planning to threat monitoring and legislative developments are essential for realising the full benefits of the ECCF.

Finally, the evaluation identifies opportunities for regulatory simplification and burden reduction. Streamlining administrative processes, clarifying terminology and harmonising requirements across legislative acts can help reduce unnecessary complexity and improve efficiency. Stakeholders emphasised the value of voluntary approaches, clear guidance documents and coordinated implementation to avoid overlaps and ensure coherence.

Some findings, such as the long-term impact of new certification schemes and the effectiveness of market surveillance, are too preliminary to draw firm conclusions and warrant a ‘wait and see’ approach. Ongoing monitoring and stakeholder engagement will be necessary to assess whether these issues resolve themselves over time or require further intervention.

1. Lead DG, Decide Planning/CWP references

Lead DG: Directorate-General for Communications Networks Content and Technology (CNECT).

Decide: PLAN/2023/181

Evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework

2. Organisation and timing

14 July 2023 - 4 December 2024 – The call for evidence was launched on 14 July 2023 (see below) and the final study report was delivered on 4 December 2024 (see below).

3. Consultation of the RSB

N/A

4. Evidence, sources and quality

Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework according to Regulation (EU) 2019/881 – STUDY 2023/037, carried out for the European Commission by PwC EU Services EESV, Open Evidence S.L. and PPMI Group, UAB – Final report 4 December 2024

A call for evidence on the impact, effectiveness and efficiency of ENISA’s mandate and of the provisions of the European Cybersecurity Certification Framework (ECCF) was conducted from 14 July 2023 to 16 September 2023. The call for evidence was intended to elicit feedback from relevant stakeholders involved in the cybersecurity domain, and, more specifically, from those involved in the EU certification process. Overall, 41 stakeholders contributed to the call for evidence, mostly from the private sector.

Provide a critical assessment of the work carried out by the external contractor which allows an understanding of why you agreed or disagreed with their conclusions.

This annex provides a detailed account of the methodology and analytical approaches used in the evaluation of ENISA and the ECCF, as documented in the 2024 ENISA and ECCF Evaluation Report. This annex explains the data collection process, sources and stakeholder consultation activities, as well as the analytical techniques and validation steps applied throughout the evaluation. It also addresses any changes from the original plan, known limitations and the measures taken to ensure the reliability and robustness of the findings. The information presented here is intended to ensure full transparency and to support the credibility of the conclusions of the evaluation.

1. Overview and process

The evaluation of ENISA and the ECCF was conducted through a comprehensive, multi-method approach designed to ensure transparency, robustness and stakeholder engagement at every stage. The study was carried out by an external contractor, with close involvement and oversight from the European Commission. All planned activities, including surveys, interviews, workshops and desk research, were completed as scheduled, with only minor adaptations to the original plan as set out in the evaluation roadmap. Where challenges arose, such as difficulties in engaging certain stakeholder groups, mitigating measures were implemented to ensure the integrity and completeness of the evidence base.

2. Data collection and sources

Desk research and administrative data

Administrative and monitoring data provided by ENISA and the Commission formed the cornerstone of the evaluation. These data were used to assess the efficiency of ENISA, triangulate stakeholder perspectives and provide a factual basis for answering the evaluation questions. Internal documents, including minutes, reports and audits, were systematically reviewed to complement and validate findings from other sources.

For the ECCF, the evaluation focused on legislative texts, notably the Cybersecurity Act and its amendments, as well as new legislative documents such as the NIS2 Directive, CRA and the Union Rolling Work Programme (URWP). The Commission’s call for evidence on the ECCF was also analysed, with particular attention to feedback from SCCG members.

Document and literature review

The literature review drew primarily on official documents published by ENISA and the European Commission, including eleven official reports, three studies and relevant website publications. Annual activity reports with annexes were extensively used to assess ENISA’s performance. For certification schemes, published draft schemes for the EUCC and EUCS were reviewed to understand the state of play and barriers to adoption.

Academic literature review

Academic research was used to situate the evaluation within the broader European cybersecurity debate and to inform the initial focus of data collection. The literature provided critical perspectives on ENISA’s epistemic authority, the challenges and vulnerabilities of the European cybersecurity domain and the alignment of the ECCF with international standards and best practices. Comparative studies with other certification schemes were also reviewed to assess the competitiveness and adaptability of the ECCF.

3. Stakeholder consultation

Interview programme

The interview programme was a central pillar of the evaluation, designed to capture a wide range of perspectives from across the EU cybersecurity ecosystem. In total, 182 individuals were contacted (40 for ECCF, 142 for ENISA), resulting in 52 interviews for ENISA and 13 for ECCF. All interviews included questions on both ENISA and ECCF and 19 stakeholders were selected from survey respondents who volunteered for interviews.

ENISA: 52 interviews were conducted, involving ENISA staff, representatives from DG CNECT, DIGIT, INTPA, NEAR, other EU entities, industry, academia, international representatives and Member State officials.

ECCF: 13 interviews were conducted with 31 interviewees, including EU institutions, ECCG members and SCCG members. Despite challenges in engaging private stakeholders (only three of nine SCCG members responded), the analysis of stakeholder consultation data helped offset this limitation.

Survey programme

The survey approach was adapted after the inception report: instead of two separate surveys for internal and external stakeholders, an integrated survey was conducted to address all stakeholder groups. The survey included specific sections on ENISA and the ECCF, with question filtering and branching to ensure relevance.

·Design: The survey was developed based on desk research and initial interviews. It included both closed and open questions, with a limited number of mandatory items.

·Dissemination: Conducted via EUSurvey, 856 stakeholders received a personal invitation. The Commission and ENISA promoted the survey through various channels and the deadline was extended to maximise responses.

·Results: 209 responses were collected, with 70 respondents (33%) involved with the ECCF. The survey results were analysed using data science and analytics software, supplemented by manual analysis.

4. Workshops and validation activities

SWOT and recommendations workshops

SWOT workshop: held online on 21 May 2024 with 32 participants, including experts from academia, ENISA and the Commission. The workshop aimed to develop a SWOT analysis for ENISA and the ECCF, validate preliminary findings and foster technical exchange. Interactive polling and breakout sessions were used to gather and validate stakeholder input. The results were used to refine the strengths, weaknesses, opportunities and threats identified for both ENISA and the ECCF.

Recommendations workshop: held online on 12 July 2024 with 77 participants. The workshop presented preliminary evaluation results and facilitated collaborative discussions on lessons learned and potential improvements for ENISA and the ECCF. Interactive polls and an anonymous form to be filled by each Member State were used to collect feedback and validate findings.

Validation and quality assurance

The evaluation process included multiple validation steps to ensure the reliability and robustness of the findings:

·preliminary findings were presented to stakeholders in workshops, where interactive polling and open discussion allowed for real-time validation and refinement;

·an anonymous asynchronous feedback form was provided to capture additional input from stakeholders unable to participate in real time;

·the iterative process of presenting, discussing and refining findings ensured that the analysis accurately reflected the collective insights and perspectives of all participants.

5. Analytical approaches and quality assurance

The evaluation combined qualitative and quantitative methods, including desk research, literature review, stakeholder interviews, surveys and workshops. Data analysis was supported by analytics software and manual review. Quality was ensured through:

·iterative validation with stakeholders and expert input during workshops;

·systematic triangulation of data sources to cross-check findings;

·adaptation of tools (e.g. survey design) to maximise relevance and response rates.

6. Limitations and mitigating measures

·Data and timing: the main limitation was the difficulty in securing interviews with private stakeholders, particularly SCCG members. This was mitigated by analysing stakeholders’ consultation data and extending the survey deadline.

·Reliability: the use of multiple data sources and triangulation of findings enhanced the reliability of the evaluation. Validation through stakeholder workshops and polling further strengthened the robustness of the conclusions.

·Uncertainty: some results, especially regarding the ECCF (where only one scheme was adopted by early 2024), are preliminary. The robustness of findings was enhanced by cross-referencing multiple sources and validating with stakeholders, but some uncertainty remains due to the evolving nature of the policy landscape.

7. Points of comparison

The main point of comparison for the evaluation was the baseline scenario of 2017, reconstructed from the 2017 Impact Assessment for the Cybersecurity Act and supporting studies. This baseline included ENISA’s status and resources, the certification landscape, market and economic data and stakeholder perceptions. All comparisons and assessments were made against this baseline, as set out in the original evaluation roadmap.

8. Critical assessment of the contractor

The contractor’s evaluation of ENISA demonstrated a certain positive bias, largely attributed to interviews with ENISA staff constituting a significant part of the interviews overall. While the report rightly acknowledges ENISA’s resource constraints, the analysis could be more balanced regarding other areas for improvement. The study was improved following feedback from DG CONNECT in several instances.

This annex presents the evaluation matrix that serves as the central organising framework for the assessment of ENISA and the ECCF. It provides detailed, criterion-based responses to the evaluation questions, covering all five evaluation criteria: effectiveness, efficiency, coherence, relevance and EU added value.

The evidence and analysis included in this annex substantiate the findings outlined in Section 4 of the main evaluation report. For clarity and transparency, the questions and their corresponding evidence-based answers are presented individually and grouped by evaluation criterion. The depth of coverage for each criterion reflects its relative importance and the extent of supporting evidence available in the main body of the report.

Findings are presented separately for ENISA and the ECCF to reflect their distinct roles and contributions within the broader cybersecurity landscape.

1. Evaluation matrix:

The following sections present refined and up-to-date evaluation matrices used to assess the effectiveness, impact, efficiency, coherence, relevance and EU added value of both ENISA and the ECCF. These matrices are aligned with the latest edition of the European Commission’s Better Regulation Guidelines and outline the evaluation questions (EQs) and operational questions (OQs), along with the corresponding judgement criteria, stakeholder groups and indicators used to evaluate each criterion.

The matrices also specify the data sources required to determine the values of the proposed indicators. Certain EQs and OQs have been specifically designed to address aspects unique to the ECCF.

Functioning as a structured analytical tool, the evaluation matrices demonstrate how evidence collected through various methodologies and data sources contributes to answering the evaluation questions and shaping the overall conclusions.

Effectiveness and impact

In line with the Better Regulation Toolbox, the effectiveness analysis examined how successfully ENISA and the ECCF have achieved or made progress toward their intended objectives. This involved assessing the actual outputs, results and impacts in relation to the goals of the action.

The effectiveness analysis yielded insights into the progress made so far by ENISA and the ECCF. The evaluation drew heavily on the intervention logic of both entities.

The tables below illustrate the structure of the evaluation matrix. Where appropriate, findings are presented separately for ENISA and the ECCF.

ENISA

ECCF

Efficiency

Efficiency-related questions examine the resources invested in relation to the changes generated by the measure. This involves assessing the inputs against the outputs, results and impacts, essentially weighing costs against benefits. The efficiency evaluation explored the costs associated with the EU measure as they affect various stakeholder groups, along with the factors influencing these costs and their connection to ENISA and the ECCF.

This evaluation was conducted based on the outcomes experienced by different stakeholder groups, as identified in the effectiveness analysis. Where feasible, these outputs were quantified. The analysis also acknowledged that while costs may initially outweigh benefits, net benefits could emerge over time, particularly given ENISA’s recently extended mandate and the early-stage development of the ECCF.

The overall conclusions drawn from the efficiency questions provided insight into whether the resources allocated to ENISA’s activities and the implementation of the ECCF are being used optimally and are necessary to achieve the measure’s objectives. Where findings indicated inefficiencies, opportunities for simplification and burden reduction were identified and examined. Where applicable, the evaluation also highlighted areas with the potential for improving efficiency and streamlining the design and implementation of the measure.

The tables below outline the structure of the evaluation matrix. Findings are presented separately for ENISA and the ECCF.

ENISA

ECCF

Coherence

The coherence analysis examined the extent to which the objectives of ENISA and the ECCF align with and complement other initiatives and the work of EU and national bodies in the field of cybersecurity. Specifically, the external coherence analysis assessed how well ENISA and the ECCF support the broader cybersecurity policy goals of the European Commission.

Internal coherence was also evaluated to determine how effectively the various components within ENISA and the ECCF function together to achieve their respective objectives.

The tables below present the structure of the evaluation matrix. Where relevant, findings are presented separately for ENISA and the ECCF.

ENISA

ECCF

EU added value

In accordance with the Better Regulation Toolbox, the assessment of EU added value focused on identifying changes resulting from the activities of ENISA and the ECCF that would not likely have occurred through actions taken solely by Member States. The evaluation considered the factors contributing to EU added value, such as enhanced coordination, improved effectiveness or efficiency and reduced administrative burden, among others.

The tables below outline the structure of the evaluation matrix. Findings are presented separately for ENISA and the ECCF.

ENISA

ECCF

Relevance

According to the Better Regulation Toolbox, the relevance evaluation compared the needs and challenges present at the time of the adoption of Regulation (EU) 2019/881, which established ENISA’s current mandate and the European cybersecurity certification scheme, with those encountered during its implementation. The evaluation also examined how the current and anticipated future needs and problems within the EU align with the objectives of ENISA and the ECCF.

The relevance evaluation identified potential mismatches between the objectives of ENISA and the ECCF and the evolving cybersecurity landscape. For instance, some of the ‘problem drivers’ outlined in the original impact assessment may no longer be applicable, while emerging technological developments could introduce new challenges related to cybersecurity innovation.

The tables below outline the structure of the evaluation matrix. Findings are presented separately for ENISA and the ECCF.

ENISA

ECCF

Annex IV provides an overview of the costs and benefits projected in the preferred options of the 2017 CSA IA. It also shows the potential simplification and burden reduction savings identified in the current 2025 IA for the updated CSA preferred options. This annex presents a comparison, highlighting both the initial expectations and the most recent projections for the main policy options considered for ENISA and the ECCF. It is important to note that, at the time of this evaluation, there is no quantitative data available on the simplification or burden reduction already achieved for ENISA. For certification, no realised savings can be reported yet, as the first EU-wide certification scheme was only adopted in May 2025 and the evaluation was conducted between 2023 and 2024. As a result, the annex focuses on projected and potential impacts, rather than on realised monetary benefits or cost reductions for either ENISA or the ECCF.

This annex summarises all the consultation activities carried out as part of the evaluation of ENISA and the ECCF. The consultation strategy was designed to collect input from all relevant stakeholder groups, including public authorities, EU institutions, industry, academia, civil society and individual citizens. It covers the full range of activities undertaken, such as the call for evidence, the targeted survey, targeted interviews and dedicated workshops. It provides a consolidated overview of stakeholder feedback on the performance, strengths, weaknesses and areas for improvement of ENISA and the ECCF. The evidence collected through these activities forms a key part of the analytical framework supporting the evaluation and informs the recommendations for future policy development.

1. Consultation scope and objectives

Scope

The stakeholder consultation activities were undertaken as part of the evaluation of ENISA and the ECCF, in accordance with Article 67 of Regulation (EU) 2019/881, known as the Cybersecurity Act. The evaluation covered the period from 2017 to 2023 and sought to assess the performance of ENISA and the ECCF against the criteria of effectiveness, efficiency, relevance, coherence and EU added value. The consultation activities were designed to gather evidence from a wide range of stakeholders in order to inform the evaluation and support evidence-based policy-making.

The consultation addressed both ENISA and the ECCF, focusing on ENISA’s mandate, objectives, governance, working practices and the implementation of the ECCF. The scope included assessing ENISA’s support for policy development and implementation, capacity building, stakeholder engagement, and the development and adoption of cybersecurity certification schemes under the ECCF.

Objectives

The objectives of the stakeholder consultation were as follows:

·to collect comprehensive and representative feedback from all relevant stakeholder groups regarding the performance, strengths, weaknesses and areas for improvement of ENISA and the ECCF;

·to assess the effectiveness, efficiency, relevance, coherence and EU added value of ENISA’s activities and the ECCF, as perceived by stakeholders;

·to identify lessons learned and recommendations for potential changes to the existing Regulation and for improving the performance of ENISA and the ECCF;

·to ensure that the evaluation is informed by the experiences, needs and expectations of Member States, EU institutions, industry, standardisation bodies, academia, non-governmental organisations and other relevant actors;

·to gather input on the implementation and impact of the ECCF, including the development and adoption of certification schemes, stakeholder involvement, transparency and alignment with EU and international standards.

The consultation was structured in such a way as to provide evidence for the evaluation’s analytical framework, which included a mixed-methods approach comprising desk research, surveys, interviews and workshops. The consultation aimed at systematically collecting and analysing the perspectives of all key stakeholders, including public authorities, the private sector, civil society, members of academia, experts and individual citizens, to inform the evaluation conclusions and the policy-making process.

2. Mapping of stakeholders

The consultation activities to evaluate ENISA and the ECCF were designed to ensure broad and balanced representation of all relevant stakeholders in the European cybersecurity landscape. Stakeholders were identified and mapped according to their institutional role, their involvement in cybersecurity policy and practice, and their relationship to ENISA and the ECCF. This mapping informed the design and targeting of surveys, interviews and workshops.

The main stakeholder categories included:

·EU institutions and bodies: this group comprised the European Commission, the European Parliament and decentralised agencies with responsibilities in cybersecurity policy, oversight and implementation. Their input focused on policy development, regulatory coherence and cross-border coordination.

·National public authorities: national cybersecurity agencies, competent ministries, regulators and other authorities responsible for implementing and enforcing cybersecurity policy were included. These stakeholders provided perspectives on national approaches, regulatory challenges and the practicalities of executing policy.

·Industry and private sector organisations: this category included companies involved in developing, providing or operating digital products and services, such as hardware manufacturers, software developers, cloud service providers and cybersecurity solution vendors. Their input was essential for understanding the impact of certification schemes and regulatory requirements on market stakeholders.

·Industry associations and representative bodies: associations representing the collective interests of businesses and industry sectors were engaged to reflect the perspectives of both large enterprises and SMEs. These organisations often acted as intermediaries, helping their members participate in the consultation process.

·SMEs: these were consulted both directly and through representative associations. Their feedback was important for assessing the proportionality and accessibility of the regulatory framework, given their specific resource constraints and operational realities.

·Academic and research institutions: universities, research centres and think tanks with expertise in cybersecurity, standardisation and policy evaluation contributed analytical and technical perspectives. Their involvement ensured a robust assessment of the strengths and limitations of the current framework.

·Consumer and civil society organisations: organisations advocating for consumer rights, privacy, digital security and broader societal interests were included to ensure that the public interest was reflected in the evaluation. Their contributions addressed issues such as transparency, user protection and the societal impact of cybersecurity measures.

·International organisations and standardisation bodies: relevant international entities and standardisation organisations were engaged to provide input on how well European initiatives were aligned with global standards and practices.

·Individual citizens: members of the public were invited to participate, particularly through the targeted survey, to capture user experiences, perceptions of cybersecurity risks and expectations regarding digital security and trust.

This mapping ensured that the consultation strategy was inclusive and balanced, avoiding overreliance on any single group. It also guided the development of survey instruments and interview protocols, ensuring that questions were tailored to the specific roles and expertise of each stakeholder category.

3. Consultation activities

Consultation activities were conducted to support the evaluation of ENISA and the ECCF. The objective was to collect robust and representative evidence from all relevant stakeholder groups, ensuring that the evaluation was informed by a wide range of perspectives and experiences.

The main consultation activities included:

·Call for evidence: a call for evidence was organised to collect feedback from a wider audience, including stakeholders not directly targeted by the survey or interviews. The call for evidence was open from 14 July to 16 September 2023 and received 41 contributions from a diverse range of stakeholders, including business associations, companies, public authorities, consumer organisations, NGOs and individual citizens. The targeted survey provided additional perspectives on the effectiveness, efficiency and impact of ENISA and the ECCF.

·Survey: a targeted survey was conducted using the EUSurvey platform. The survey was designed to involve all stakeholder groups, including those involved with ENISA and the ECCF. It included both closed and open questions, with filtering to ensure that it was relevant to different respondents. The survey was sent to 856 stakeholders and was also promoted by the European Commission, ENISA and relevant associations. The survey was open from 13 February to 5 March 2024, with extensions to maximise participation. In total, 209 responses were collected, covering a broad spectrum of stakeholder categories, such as national authorities, industry stakeholders, academic institutions, consumer organisations and EU institutions.

·Interviews: a structured interview programme was carried out to gather in-depth qualitative insights. The study team contacted 182 individuals and conducted 49 interviews for ENISA and 13 for the ECCF. Interviewees included ENISA staff and representatives of the European Commission, national authorities, industry, academia and international organisations. The interviews were designed to explore key evaluation questions in greater detail and to validate findings from other consultation activities.

·Workshops: two main workshops were held for collective discussion and validation of findings. The first was a SWOT and recommendations workshop, which brought together stakeholders from academia, ENISA, the European Commission and industry to discuss strengths, weaknesses, opportunities and threats related to ENISA and the ECCF. This workshop included interactive polling and breakout sessions to gather detailed feedback and suggestions for improvement. The second workshop focused on validating preliminary evaluation results and collecting recommendations for future improvements. Participants included representatives from the European Commission, Member States and other key stakeholder groups.

The overarching objective of these consultation activities was to ensure that the evaluation of ENISA and the ECCF was informed by stakeholder input. Each activity was designed to capture a specific type of evidence: desk research provided context and baseline data; the survey programme enabled quantitative analysis of stakeholder views; interviews offered qualitative depth and validation; and workshops allowed collective reflection and consensus-building. Together, these activities ensured that the evaluation reflected the experiences, needs and expectations of all relevant stakeholders in the European cybersecurity ecosystem.

4. Call for evidence

The call for evidence was conducted from 14 July to 16 September 2023 to collect stakeholder feedback on the impact, effectiveness and efficiency of ENISA’s mandate and the ECCF. The consultation aimed to gather views from a broad range of stakeholders, particularly those involved in the EU cybersecurity certification process. In total, 41 responses were received from 13 EU Member States and two non-EU countries, with the majority of contributions coming from the private sector. Nearly half of the respondents were business associations (20), followed by companies and businesses (9), public authorities (4), consumer organisations (1), NGOs (1), research centres and standards associations (4), and individual citizens (2). Most respondents (85%) were based in the EU, with the remainder from the USA and the UK. Sectoral representation was also diverse, with significant input from digital service providers, digital infrastructure providers, manufacturing trade associations and generalist organisations.

The call for evidence sought to assess three main areas: the overall organisation and performance of ENISA, the functioning of the ECCF and feedback on specific certification schemes, particularly the Cloud Services Scheme (EUCS). Stakeholder feedback on ENISA was generally positive, with around 30% of respondents recognising ENISA as a leading centre of cybersecurity expertise in the EU. Particularly appreciated were its contributions to cyber resilience, incident response and the promotion of cooperation and best practice exchange. However, almost 22% of respondents said that ENISA’s resources and capabilities needed to be distributed adequately and consistently, citing also challenges in recruiting qualified staff. Stakeholders recommended increasing cooperation with academic and research institutions to maintain specialised expertise.

A recurring theme in the feedback was the need for greater transparency and stakeholder involvement in ENISA’s processes. Over 40% of respondents expressed concerns about insufficient engagement, particularly for smaller organisations and civil society representatives. Stakeholders called for more meaningful participation and improved communication, and for ‘update only’ meetings to be avoided. While it was highlighted that ENISA engaged with Member States through the national liaison officers network, some respondents noted limited benefits from existing working groups due to organisational challenges.

Regarding the ECCF, stakeholders generally viewed EU cybersecurity certification as a useful and promising tool, but highlighted several areas for improvement. Delays in adopting certification schemes and the Union rolling work programme (URWP) were frequently mentioned as limiting the impact of the framework. Approximately 34% of respondents reported that they were insufficiently involved in developing certification schemes). A similar proportion (32%) highlighted a lack of transparency in ECCF procedures, including opaque decision-making, insufficient information sharing and changes to draft schemes without prior consultation. Stakeholders emphasised the importance of increasing transparency regarding the composition and functioning of ad hoc working groups, and of ensuring that members have access to draft schemes and are promptly informed of significant changes.

The mandatory nature of certification schemes was another area of debate. Of the 34% of respondents who raised this issue, the majority were opposed to mandatory requirements, citing concerns about increased costs and barriers to the internal market, particularly for SMEs. Those in favour of mandatory certification stressed the need for sectoral approaches, alignment with international standards and the use of impact assessments to evaluate the potential effects on economic operators and the internal market.

The Cloud Services Scheme attracted particular attention, with 9 out of 10 stakeholders concerned about the certification process being politicised, especially regarding data localisation requirements. Respondents warned that such requirements could disrupt business relationships and hinder market competition, particularly for providers headquartered outside the EU.

Stakeholders also stressed the importance of aligning EU certification schemes with international standards (21% of respondents) and ensuring coherence with the broader EU legislative framework (20%). Recommendations included streamlining compliance, allowing audit reports to be reused for multiple legal acts and ensuring mutual recognition of cybersecurity certificates across EU countries.

In summary, the call for evidence highlighted both strengths and areas for improvement in ENISA’s mandate and in the ECCF. Stakeholders recognised ENISA’s expertise and positive impact, but called for adequate resources that are distributed appropriately, for greater transparency and for more inclusive stakeholder engagement. As regards the ECCF, the feedback underscored the need for schemes to be adopted in a timely manner and for greater transparency, stronger stakeholder involvement and alignment with international standards and EU law. The consultation also revealed concerns about certification processes being politicised by considerations related to non-technical risk factors. Overall, the feedback emphasised the importance of collaborative and inclusive approaches to developing effective European cybersecurity policies and certification frameworks.

5. Targeted survey

The targeted survey formed a central part of the evaluation of ENISA and the ECCF. It aimed to collect evidence and perspectives from a broad pool of stakeholders across the European cybersecurity ecosystem. The consultation was designed to assess the effectiveness, efficiency, relevance, coherence and added value of ENISA and the ECCF, and to ensure that the evaluation was informed by the experiences and expectations of all relevant stakeholders. This included stakeholders directly involved with ENISA and the ECCF, as well as those who benefit from their outputs and activities.

The survey programme was conducted via the EUSurvey platform, with the questionnaire developed on the basis of desk research, initial interviews and feedback. The survey included both closed and open questions, with filtering and branching logic to ensure that respondents were asked questions relevant to their background and involvement with ENISA and/or the ECCF. The survey was launched on 13 February and closed on 5 March 2024, with extensions and follow-up reminders to maximise participation.

5.1 Stakeholder participation

A total of 856 stakeholders received a personal invitation to complete the survey. The survey link was also promoted via the Commission’s news page, in ENISA’s internal communications and by the relevant associations. Up to three follow-up emails were sent to non-respondents and the deadline was extended by one week to accommodate additional responses. In total, 209 responses were received, representing a broad cross-section of stakeholder groups and ensuring a robust and representative evidence base for the evaluation.

Table 1. Number of survey responses per stakeholder group (overall)

ECCF-specific stakeholder participation

Of all the respondents, 70 (33%) indicated that they were involved with the ECCF and received a specific set of questions about its implementation. The breakdown for ECCF stakeholder groups is as follows:

Table 2. Number of survey responses per stakeholder group (ECCF)

Note: Respondents could select more than one option.

5.2 Summary of results

ENISA survey results

Effectiveness

ENISA is generally acknowledged by stakeholders as effective in fulfilling most of its mandate. According to the survey, 71% of stakeholders considered ENISA a leading centre of expertise on cybersecurity, able to deliver valuable outputs, including for policy-making and decision-making processes. ENISA’s support was particularly appreciated during critical periods, such as the COVID-19 pandemic, and for its operational cooperation with Ukraine.

ENISA’s publications were cited most frequently by stakeholders in the cybersecurity field, confirming ENISA’s status as a centre of expertise in this area. Between 2017 and 2023, ENISA produced and issued a total of 286 publications. These covered a wide range of topics, with the most frequent being cybersecurity policy (72 publications), cyber threats (48), critical infrastructure (38), incident reporting (24) and emerging technologies (21). Stakeholders were involved in the preparation of these publications through workshops and studies. While these publications were highly cited and appreciated for their independence and clarity, many respondents noted in the open-ended follow-up questions that these could be made more concise and practical. There was a clear call for using summaries and visualisations more widely to make key information easier to identify.

Supporting policy implementation is one of ENISA’s key tasks, as reinforced by the Cybersecurity Act. ENISA’s support has contributed to its stakeholders adopting regulatory or policy changes and innovations. Respondents provided concrete examples, such as adopting structured threat communication protocols and using ENISA’s guidance on implementing the NIS Directive to improve cybersecurity resilience. Respondents also underlined the influence of ENISA’s frameworks on national cybersecurity certification schemes and the fact that ENISA’s foresight activities helped anticipate future regulatory needs. Over 80% of ENISA’s contribution was related to organising workshops and conferences. However, according to survey results, only about half of national authorities and companies consistently used ENISA’s outputs for policy- or decision-making processes.

Figure 1 ENISA’s outputs in policy tasks

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 21 ‘Please indicate to which extent ENISA’s activities and outputs contributed to the following aspects? My organisation uses ENISA’s outputs in its policy and or decision-making processes’.

The consultation results also show ENISA’s effectiveness in supporting Member States through the Cybersecurity Support Action. 58 % of all stakeholders agreed that ENISA effectively supported Member States in preventing and responding to cyberattacks through that Action, with no notable differences between various stakeholder groups. However, the consultation also revealed that 44% of survey respondents were unable to assess the usefulness of the Action, indicating that the visibility of the Action needed to be improved.

Figure 2 The Cybersecurity Support Action’s support to Member States in preventing and responding to cyber attacks

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 25 ‘In your opinion, to which extent has the ENISA Cybersecurity Support Action been effective in supporting EU member states in preventing and responding to cyberattacks?’

In summary, while stakeholder satisfaction with ENISA’s approach and activities is generally positive, given its valuable outputs and role as a centre of expertise in cybersecurity, there are areas where its relevance can be significantly improved. ENISA’s contributions to policy implementation and capacity building are appreciated by stakeholders, although there are clear opportunities for improvement in communication, stakeholder engagement and resource allocation.

Efficiency

During the 2017-2023 evaluation period, stakeholders were enthusiastic about ENISA’s performance, praising it for successfully delivering its outputs even during periods of high workload, and recognising its operations as mostly efficient under its existing governance structure. Despite this, the evaluation and survey highlighted several key areas where stakeholders found that ENISA still had room to become more efficient.

Responses to open-ended questions showed that some stakeholders were confused about ENISA’s role within the EU’s cybersecurity framework. Notably, 28% of the respondents surveyed were more restrained with regard to seeing ENISA as the centre of expertise on cybersecurity. They noted that the lack of a more effective communication system jeopardises ENISA’s standing as the centre of expertise and that the complicated structure of ENISA’s website prevented them from interacting properly with its outputs.

Interviews with ENISA’s staff, stakeholder surveys and internal documentation indicated that ENISA struggled to keep pace with increasing demands and struggled to fill specialised positions, exacerbated by a global shortage of IT and cybersecurity specialists. This led to delays, reprioritisation of tasks and periods of high stress and workload. Periods of high workload for ENISA were often associated with adopting and implementing new policies or legal acts, with developing cybersecurity certification and with operational activities related to geopolitical developments. However, 63% of stakeholders gave a positive assessment of ENISA’s performance (‘successful’ or ‘very successful’) during such periods. The survey results indicate that ENISA’s organisational arrangements were relatively well adapted to managing periods of high workload. A total of 84% of stakeholders ‘completely’, to a ‘large extent’ or ‘somewhat’ agreed with this statement.

However, resources and resource allocation, operational inefficiencies and challenges stemming from the political and regulatory environment were identified as the main obstacles to ENISA’s performance during periods of high workload. The 2022 Staff Satisfaction Survey shows that 64% of respondents experienced stress due to high workload.

In 2023, an average of 76% of staff reported working more than 40 hours per week monthly and in 2022, 4 FTEs resigned due to overwork and work over weekend. Some stakeholders noted that more resources and, to some extent, a more agile approach to deploying resources could be a way for ENISA to better adapt to evolving cybersecurity demands and to minimise delays. Other stakeholders thought that ENISA could increase its capacity to provide policy and technical support by being more selective with its engagements and refining its operational focus areas.

Stakeholders also noted that budget management presents opportunities for improvement. Despite significant budget growth from 2017 to 2023, resource constraints persisted. ENISA’s budget grew unevenly, with notable increases in 2019 (over 46%), 2020 (30.5%) and 2022 (72.4% compared to 2021, due to the Cybersecurity Support Action). ENISA encountered a downward trend in balancing approved and committed appropriations between 2019 and 2022 due to delays in actions like the Cybersecurity Support Action. Reversing this trend and dedicating efforts towards managing administrative expenditure, including addressing procurement delays, could further improve internal efficiency.

In summary, ENISA demonstrated efficiency in implementing its tasks, supported by a revised governance structure. However, stakeholders underlined opportunities for ENISA to become more efficient by improving its communications and by making more resources available or allocating resources in a more strategic manner.

Relevance

ENISA’s relevance within the cybersecurity domain is shown by its responsiveness to evolving stakeholder needs and its flexibility to adapt to the changing landscape. According to stakeholders, it has consistently demonstrated its ability to review and realign its areas of action in response to emerging developments. Thus, it continues to be a vital part of the EU’s cybersecurity framework.

The results of the survey confirmed that ENISA’s work was mostly relevant to stakeholders’ needs. It regularly reviewed its areas of activity to respond to emerging needs and remained agile by setting up ad hoc working groups. Stakeholders expressed a high level of satisfaction with how ENISA responded to changes in the cybersecurity landscape, although some felt that it needed more tools to effectively address the growing cyber threats in Europe. National cybersecurity authorities and smaller Member States benefited from ENISA’s capacity building initiatives and regulatory insights. However, stakeholders noted that ENISA could further increase its relevance in some areas. To do this, they suggested that it improve its support and increase its visibility among various sectors and stakeholders, especially SMEs, which often find it hard to meet cybersecurity requirements. Stakeholders also called for more sector-specific tools and resources, as well as insights and tools to tackle emerging threats. 

The stakeholders surveyed reported being satisfied with ENISA’s ability to adapt to the changing cybersecurity landscape. Specifically, 74% of respondents mostly or strongly agreed that ENISA quickly adapted to changes in the cybersecurity landscape in 2019-2023, and 70% mostly or strongly agreed that ENISA sufficiently addressed all major unforeseen cybersecurity incidents during that period. There were no notable differences between the responses of different stakeholder groups.

While stakeholders acknowledged ENISA’s efforts, some respondents felt that it needed greater capabilities to effectively address all cyber threats in Europe. Respondents argued that ENISA lacked an effective incident response unit and operational capabilities to act quickly in the face of cybersecurity incidents. They felt that these were not sufficiently supported by ENISA’s mandate and governance structure. Some respondents stressed that directly managing cybersecurity incidents was not the purpose of ENISA and that it had no capabilities in this area. Instead, ENISA supports Member States in dealing with incidents, and those respondents pointed to successes in this context with the CSIRTs network. ENISA contributed to the EU’s preparedness to face cyber threats by supporting and organising cyber exercises and by improving operational cooperation between Member States, including through its support to the CSIRTs network.

ENISA set up appropriate practices for stakeholder consultation and management, however industry stakeholders expressed dissatisfaction with the consultation and collaboration processes that involved them, as well as with the difficulty in accessing information.

While ENISA’s outputs and services generally align with stakeholder needs, notable discrepancies exist in how different groups of stakeholders perceive ENISA’s responsiveness to their needs. ENISA’s support is perceived as most relevant by EU institutions, and relevant to a lesser extent by national authorities or entities and private bodies. The difference in the relevance of ENISA’s work to its stakeholders was underlined by the varying level of satisfaction with its services and outputs. 44% of all respondents indicated that their needs were met only ‘somewhat’, ‘to a small extent’ or ‘not at all’. This figure rises to 50% for respondents representing industry organisations, with a comparatively higher proportion of respondents selecting ‘to a lesser extent’. This data suggests that there is still room for ENISA to tailor its efforts to better meet the needs of national organisations and private bodies.

Figure 3 Relevance of ENISA’s support to different groups of stakeholders

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 21 ‘Please indicate to which extent ENISA’s activities and outputs contributed to the following aspects: ENISA’s outputs and services correspond to my organisation’s needs’.

National cybersecurity authorities and agencies valued ENISA’s role as a key facilitator of cooperation and information exchange between Member States, as well as in promoting technical cooperation and a common standard of cybersecurity. They also valued the insights provided by ENISA on new regulations, guidelines, best practices and national requirements. Capacity-building initiatives were particularly appreciated by smaller Member States with more limited internal capabilities, while support for common standards proved particularly beneficial for EU candidate countries.

Representatives of international organisations, industry and vendors highlighted the importance of ENISA in representing the EU’s position in the increasingly complex regulatory landscape of cybersecurity compliance and technical policy. ENISA’s role in facilitating cooperation between different types of national and international stakeholders was described as crucial for ensuring a common understanding among different stakeholders. However, respondents called for ENISA to play a more direct role in providing tools and support to different sectors in order to increase its visibility and impact. ENISA could improve support for SMEs across the Digital Single Market to help such companies better integrate and comply with cybersecurity standards.

ENISA’s support to EU institutions was well received, including its support in promoting situational awareness and crisis management. Its role in collecting information and liaising with Member States on cybersecurity was also valued. Respondents from the Commission expressed confidence in ENISA as a reliable implementing partner and a source of technical and operational expertise. ENISA has made contributions to national and EU policies and legislative initiatives, which have been generally well received by stakeholders. According to ENISA, its ability to provide policy support to the Commission is mainly hindered by a lack of qualified policy experts.

The role and purpose of ENISA in policy-related tasks were found to be clear and properly defined, as confirmed by 72% of the stakeholders surveyed. However, some stakeholders had expected ENISA’s role in policy-related tasks to go beyond its current mandate.

In summary, ENISA’s relevance is well established, and its flexibility, strategic alignment and stakeholder engagement are valued. Nevertheless, stakeholders stressed that there is still significant room for ENISA to better meet their needs.

Coherence

ENISA played an active role in fostering cooperation and knowledge sharing among stakeholders, as confirmed by the results of the survey. Its efforts were thought to complement those of national cybersecurity authorities and CERT-EU, although some interviewees noted that there were areas of overlap. ENISA supported key EU networks such as CSIRTs and EU-CyCLONe, organised exercises such as CyberEurope, and facilitated the exchange of best practices through the NIS Cooperation Group.

Survey respondents provided a positive assessment of ENISA’s efforts to exploit synergies in expertise and knowledge sharing with other stakeholders. Representatives of private bodies were somewhat less satisfied (65%) compared to other stakeholder groups (74% overall).

Figure 4 ENISA sufficiently exploited synergies with other stakeholders

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, questions 15 and 23: ‘Please indicate to what extent you agree or disagree with the following statements about ENISA: ENISA sufficiently exploited synergies with other stakeholders’

There were a few overlaps between the formal responsibilities of ENISA and those of other relevant stakeholders, but these activities were mostly carried out in a complementary way. Stakeholders saw the greatest overlap between ENISA and the national cybersecurity authorities, the ECCC, CERT-EU and Commission services. They identified a need to improve synergies between the responsibilities and actions of ENISA and those of other EU bodies. Responses to open-ended follow-up questions indicated that there were already complementarities with national authorities in policy-related tasks, risk management, capacity building and incident response, which were mutually beneficial. Some survey respondents underlined that, by formalising cooperation arrangements with other entities, such as EMSA and the JRC, ENISA could better leverage synergies and ensure a unified approach to cybersecurity initiatives.

Figure 5 Overlaps between ENISA and other stakeholders in the field of cybersecurity

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 26 ‘In your view, to what extent, if at all, do ENISA’s outputs and services overlap with those of the following institutions active in the area of cybersecurity?’

ENISA strengthened its cooperation with EU, regional and international stakeholders during the evaluation period. This was demonstrated by the increasing number of structured cooperation frameworks set up. Several MoUs were signed with other EU bodies to improve working arrangements and facilitate the sharing of knowledge, information and expertise. ENISA also signed a service level agreement with the ECCC in the fields of research, innovation and administration in 2022. This was followed by an MoU in 2023 to coordinate the implementation of operational tasks and research initiatives. Despite this, some survey respondents still see a need to further calibrate activities and increase coordination between both organisations.

ENISA facilitated cybersecurity cooperation at EU and national levels. Stakeholders had positive opinions of ENISA’s promotion of cybersecurity cooperation, including information sharing and coordination, during the evaluation period. National authorities assessed ENISA’s contribution to cybersecurity promotion especially positively, with 83% of survey respondents indicating that it had achieved this objective to a large extent or completely.

Figure 6 ENISA’s contribution to promoting cybersecurity cooperation

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 16: ‘In your opinion, to what extent has ENISA achieved the following objectives during the period of 2019-2023? Promoting cybersecurity cooperation, including information exchange and coordination, at EU level between MS, EU institutions, bodies, offices and agencies and relevant private and public stakeholders’

ENISA also strengthened its regional cooperation with non-EU countries, particularly with the Western Balkans and Ukraine. ENISA set up working arrangements with the US Cybersecurity and Infrastructure Security Agency (CISA) in the areas of capacity building, exchange of best practices and increasing situational awareness.

Survey respondents and interviewees expressed that ENISA’s interactions with private stakeholders and international partners must be more predictable and transparent to maintain confidence and foster collaboration. Around half the representatives of private bodies found ENISA’s role in contributing to cooperation and coordination between stakeholders to be limited. In this context, private bodies suggested that ENISA could improve its outreach and stakeholder engagement activities and could expand its partnerships with global cybersecurity stakeholders, particularly as regards collaborating with industry representatives and non-EU countries.

Figure 7 ENISA’s contribution to cooperation and coordination between stakeholders

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 21 ‘Please indicate to which extent ENISA’s activities and outputs contributed to the following aspects: The activities of ENISA have improved the cooperation and coordination between my organisation and other stakeholders’

In summary, stakeholders found that ENISA demonstrated a solid foundation in promoting cybersecurity coherence in the EU. They thought that its structured cooperation frameworks, support for key EU networks and facilitation of best practice exchanges contributed to a coordinated and coherent approach to cybersecurity across the EU. Some recommendations included addressing current inefficiencies and improving inter-Agency coordination.

EU Added Value

According to stakeholders and survey respondents, ENISA has made a significant contribution to the strengthening of the EU’s cybersecurity ecosystem. It is seen as a central hub that has encouraged vital cooperation across the EU, supported national efforts, particularly in Member States with less mature cybersecurity frameworks, and helped align cybersecurity practices and policies. At the same time, stakeholders have identified areas where ENISA’s impact could be reinforced.

ENISA’s current configuration was considered optimal for maintaining independence and facilitating close cooperation with Member States. Stakeholders emphasised that ENISA’s independence from political influence added great value.

ENISA’s role in promoting convergence and harmonisation across the European cybersecurity landscape was achieved through active engagement with Member States, including the involvement of national experts in strategic and day-to-day discussions. ENISA coordinated with bodies such as CERT-EU, Europol and EC3 to produce joint reports, avoiding duplication and improving shared situational awareness. Agreements with other EU bodies, such as the ECCC and the European Union Agency for Railways (ERA), facilitated cooperation and avoided overlapping mandates. ENISA’s secretariat roles in the CSIRTs network and EU-CyCLONe ensured continued coordination between and the effectiveness of these groups.

According to the survey, around two thirds of stakeholders considered that the collection and dissemination of relevant cybersecurity information, the generation of new knowledge, insights and evidence on cybersecurity issues and support for the implementation of EU cybersecurity policies at the national level would be hard to achieve without ENISA. Three quarters of respondents believed that improving Member States’ cybersecurity capacities, and raising awareness of cybersecurity issues, would have little effect in the absence of ENISA. Around half of respondents believed that improving the cybersecurity capacities of EU institutions would be possible only to a small extent or would not be possible at all without ENISA. As for facilitating cooperation with third countries and international organisations, 45% of respondents said this would be possible only to a small extent without ENISA and 7% said it would not be possible at all. For enhancing cybersecurity cooperation between stakeholders in the EU, 46% said this would be possible only to a small extent and 7% said not at all.

Figure 8 Achieving ENISA’s objectives without ENISA itself

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 34 ‘In your view, to what extent would the following results have been possible to achieve by Member States alone, without ENISA’s involvement?’

Most respondents said ENISA provided clear EU-level added value to the cybersecurity of the EU. ENISA’s specialised mandate, allowing it to act as a separate EU decentralised body, allowed for a dedicated and independent approach to cybersecurity. As an EU agency, ENISA’s structure and governance allows for better and easier cooperation with Member States. Its expertise continues to be important in addressing the complex and evolving threats facing the EU.

However, private entities were more critical of the idea that ENISA provided added value. For example, 6% of private bodies responded ‘not at all’ and 14% ‘to a small extent’ when asked about ENISA’s added value to their activities, compared to 0% and 8% respectively among national authorities. ENISA’s primary focus was on national authorities, making sector-specific outputs less visible and impactful. The overall impact of guidelines and events depended on the level of maturity and specific needs of the organisation ENISA dealt with. Major industry players with a longstanding commitment to cybersecurity found ENISA’s activities aligned with their routine operations but lacked new insights, indicating that more could be done to tailor insights to the private sector’s specific challenges. Companies reported that they often relied on International Organization for Standardization (ISO)/ International Electrotechnical Commission (IEC) standards rather than ENISA initiatives. Some organisations valued ENISA’s technical guidelines, tools and reports but claimed several of them were redundant in the context of existing standards. Private stakeholders monitored ENISA’s work in areas such as IoT security and cybersecurity certification to ensure consistency with international standards.

Figure 9 Added value of ENISA’s activities

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 35: ‘Overall, to what extent has ENISA provided added value to the activities of your organisation during the period of 2019-2023?’

In the event of the withdrawal of EU action, the most likely consequences were expected to be increased difficulties in achieving trans-national coordination and the development of expertise in the field of cybersecurity. Overall, according to stakeholders, abolishing ENISA would lead to coordination challenges, less effective cybersecurity measures and potentially disjointed national approaches to cybersecurity issues.

ENISA is recognised for its specific focus on cybersecurity policy implementation, stakeholder engagement and comprehensive support to Member States, which positions it uniquely within the EU’s cybersecurity landscape. While focusing on national authorities is essential, respondents said that ENISA could improve by increasing engagement with stakeholders and collaboration with industry. They also suggested that increased resources and better prioritisation could help ENISA adapt to evolving cybersecurity challenges.

ECCF survey results

Effectiveness

The ECCF contributed only somewhat to improving the cybersecurity capabilities of Member States and private companies, according to the survey. 53% of stakeholders believed that the ECCF contributed only to a small extent to enhancing the capabilities and preparedness of private companies to face cybersecurity threats, while 19% did not consider the ECCF to have made any contribution to private companies at all. For Member States, 50% of survey respondents believed that the ECCF improved their capabilities only slightly, despite the establishment of national certification authorities across all Member States. The framework was considered to fall short in its ability to support national authorities in their efforts to increase national cybersecurity certification capabilities.

The survey results also indicated that delays in the adoption of certification schemes deeply affected the ECCF’s ability to attain its objectives. Specifically, 55% of stakeholders believed that certification scheme delays negatively affected the ECCF’s ability to improve the transparency of ICT products and services. Moreover, 61% said these delays negatively impacted the public’s and businesses’ trust in and awareness of cybersecurity, and 53% believed that delays hindered efforts to reduce the fragmentation of certification schemes within the EU.

Figure 10 Objectives that were not reached according to stakeholders

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 48: ‘Do you agree with the following statements’

Despite these challenges, according to the survey, 52% of respondents reported that the ECCF improved cooperation and coordination across Member States and EU institutions, bodies and agencies.

External factors also influenced the effectiveness of the ECCF. According to the survey, 46% of stakeholders said emerging cybersecurity needs, challenges and threats had positively influenced the ECCF’s effectiveness; 42% of respondents indicated that the increase in Member States’ capabilities in cybersecurity had had a positive impact on the ECCF. However, 45% of stakeholders considered that changes in policy priorities at the EU and Member State levels had had a negative impact on the ECCF. Geopolitical factors were also significant, with 49% of stakeholders saying that they had impacted the ECCF negatively or highly negatively. The COVID-19 pandemic had had a slightly negative impact, according to 31% of survey respondents.

Figure 11 External factors influencing the ECCF’s objective

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 49: ‘In your opinion, to what extent have the following external factors positively or negatively influenced the ECCF in achieving its objectives?’

In summary, the ECCF’s effectiveness in fulfilling its objectives was limited by resource imbalances, delays in adopting schemes and external factors such as changing policy priorities and geopolitical influences. While the ECCF improved cooperation and harmonisation among Member States and EU institutions, significant challenges remained in the drive to improve certification capabilities and reduce fragmentation across the EU.

Efficiency

The efficiency of the ECCF was mostly impacted by content- and process-related issues. According to the survey, 77% of respondents reported that content issues, such as technical complexity or political impact, were the factor that most hindered the operation of the ECCF. 70% of respondents identified process-related issues, including the functioning of the preparation and adoption process, as significant obstacles to efficiency. Legal concerns, such as shortcomings in the legal framework at EU or national level, had less impact (identified by 43% of respondents).

As regards the EUCS scheme, 46% of survey respondents indicated that content issues, resulting largely from the politicisation of the debate, were the primary impediment to adoption. In the case of EU5G, 38% of stakeholders raised content-related issues and 32% attributed delays to process-related issues. The EUCC scheme was adopted in January 2024 after about 55 months of discussion, with delays attributed to technical complexity, lack of experience in developing schemes and changing policy priorities.

Survey respondents identified ENISA and its ad hoc working groups as the stakeholders that contributed most to the smooth functioning of the framework, with 77% and 88% of respondents respectively agreeing completely or to a large extent that these bodies had made a contribution. The ECCG, SCCG and NCCAs also garnered rather positive opinions, with 58% of respondents expressing a positive view for each of these stakeholder categories.

Figure 12 Stakeholders’ contribution to ensuring smooth functioning of the ECCF

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 51: ‘In your opinion, to what extent have the following stakeholders and processes contributed to ensuring a smooth functioning of the ECCF?’

According to the survey, areas of the ECCF that could be improved include the process of preparing, adopting and reviewing schemes (suggested by 81%), stakeholder’s cooperation (75%) and security objectives/assurance levels and schemes (37%).

Figure 13 ECCF improvements

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 64: ‘Which of the following areas of the ECCF, if any, could be improved?’

Given the EUCC scheme was adopted only recently, it is too early to identify the costs borne or the benefits experienced by stakeholders in terms of compliance with ECCF requirements. However, survey respondents indicated some costs and benefits related to preparatory activities, such as the development of standards, capacity building, and awareness-raising.

In summary, while ENISA and ad hoc working groups were seen as positive contributors, improvements are needed in the area of stakeholder cooperation and decision-making transparency, as well as in process management.

Relevance

The ECCF is seen as highly relevant to the achievement of internal market objectives in the current cybersecurity landscape. Stakeholders said that, given the increase in the number and intensity of cybersecurity threats, EU cybersecurity certification is a valuable asset. Key aspects that make ECCF relevant include the closer level of EU cooperation, its contribution to the development of standards and the ability to require critical infrastructures and public procurement beneficiaries to become certified. Stakeholders emphasised ECCF’s capacity to effectively improve cooperation between EU Member States and to streamline trade by providing a unified certification platform.

The ECCF’s relevance is further underscored by references to it in other EU legislation, including the Cyber Resilience Act (CRA), the NIS2 Directive and the European Digital Identity Regulation, which were all pending adoption by the co-legislators or proposed at the time of the evaluation. The CRA is expected to be crucial in promoting the certification of software and products with digital elements. The ECCF has provided input to the development of new standards and to addressing gaps in existing ones, such as for the cloud and 5G.

The mutual recognition of certified products, services and processes across the EU is still considered an effective tool to reduce individual costs for enterprises, thereby strengthening the single market and facilitating trade within the EU. The ECCF was also identified as a platform for cooperation among Member States at the EU level, aimed at fostering comprehensive evaluation and coordination.

Survey respondents confirmed the relevance of the ECCF’s scope. Specifically, 86% agreed that the scope of the ECCF is sufficient in terms of the elements within each scheme (security objectives, assurance levels and types of requirement) and 88% agreed that it is adequate to cover ICT products, processes and services.

Coherence

The ECCF has demonstrated coherence with existing cybersecurity legislation and its operations are expected to be influenced significantly by the implementation of legislative measures such as the CRA, the European Digital Identity Regulation and the NIS2 Directive, according to the survey. Some stakeholders expressed concern about the risk of potential overlaps between the CSA and the CRA; this shows that real-world integration will be a challenge and will require oversight.

Ensuring the CSA’s consistency with other EU policy and legislative measures, both cybersecurity-related (e.g. the NIS2 Directive and the CRA) and sectoral (e.g. the European Digital Identity Regulation), is essential for facilitating compliance and ensuring successful implementation of the ECCF. According to the survey, 83% of stakeholders found the framework to be coherent with existing instruments in the EU regulatory framework, with 55% answering ‘fairly coherent,’ 23% ‘very coherent,’ and 5% ‘perfectly coherent.’

A broad majority – 72% of survey respondents – agreed that the then-planned adoption of the CRA would have a significant effect on the ECCF. The CRA is expected to introduce mandatory conformity assessment of cybersecurity requirements; existing certificates under an EU scheme may offer a presumption of conformity. While the CSA and CRA legal texts seemed fairly coherent, Member States stressed that it will be important to ensure a consistent implementation of the two acts.

Figure 14 Impact of the CRA proposal on the ECCF

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 63: ‘In your view, to what extent will the impact of the ECCF increase because of the adoption of the proposed Cyber Resilience Act (CRA)?'

Despite recognising the overall coherence, more than half of respondents (54%) identified overlaps between the ECCF and other EU initiatives. Stakeholder contributions indicated that this concern stemmed from a potential overlap with the CRA, which could result in a duplication of efforts and inconsistent requirements. Member States also reported a risk of overlaps and noted the challenges involved in aligning the ECCF with international and European standardisation processes. They emphasised the need to establish a communication channel with international standardisation organisations to leverage existing European or international standards and prevent inconsistencies between standards developed at the EU and international levels.

In summary, stakeholders found the ECCF to be largely consistent with the existing EU regulatory framework, but also highlighted the importance of ensuring consistent alignment with forthcoming legislation and avoiding overlaps with other initiatives. Continued attention to terminology and standardisation processes will be necessary to maintain coherence as the framework evolves.

EU Added Value

Stakeholders broadly agreed that the ECCF brings clear EU added value compared to what could have been achieved by Member States alone. The framework’s harmonised approach to cybersecurity certification across all Member States provides a common EU baseline of security requirements for digital products, processes and services within the EU single market. This builds trust among consumers and businesses and promotes cross-border cooperation. Most stakeholders recognised the added value of the ECCF in achieving a more secure, transparent and cohesive internal market for ICT products, services and processes.

According to the survey, 84% of stakeholders believed that Member States alone could increase the uptake of certification only to some extent or to a limited extent. Furthermore, 92% of stakeholders believed that Member States alone could not have achieved more streamlined and cost-effective certification processes. For example, 41% of respondents said Member States could not have achieved more streamlined and cost-effective certification processes at all and 33% said they could have done so only to a small extent.

Figure 15 ECCF added value

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 60: ‘Do you agree with the following statements?’: ‘Due to the delay in the adoption of European cybersecurity certification schemes, Member States alone could have increased the uptake of cybersecurity certification more than what the ECCF has done’ and ‘Due to the delay in the adoption of European cybersecurity certification schemes, Member States alone could have achieved more streamlined and cost-effective certification processes than what the ECCF has done.’

When considering the benefit of the ECCF on increasing levels of trust, 88% of stakeholders agreed that Member States alone might not be able, or able only in a limited way, to increase trust and awareness among the public and businesses about the cybersecurity of ICT products, processes and services. Additionally, 92% of stakeholders agreed that Member States alone could not have increased trust and confidence in the single market more than the ECCF had done. Regarding transparency challenges, 88% of stakeholders agreed that Member States alone could have increased the transparency of ICT products and services, ensuring security by default and design, only to a limited extent. For example, 48% of respondents said Member States could not have enhanced trust and confidence in the single market at all and 28% said they could have done so only to a small extent.

Figure 16 ECCF trust and transparency added value

Source: PPMI, Intellera Consulting and PwC (2024). Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report. Survey of stakeholders, question 60: ‘Do you agree with the following statements’:’ Due to the delay in the adoption of European cybersecurity certification schemes, Member States alone could have enhanced the trust and awareness of citizens and businesses about cybersecurity of ICT products, processes and services more than what the ECCF has done’, ‘Due to the delay in the adoption of European cybersecurity certification schemes, Member States alone could have enhanced trust and confidence in the single market more than what the ECCF has done’ and ‘Due to the delay in the adoption of European cybersecurity certification schemes, Member States alone could have enhanced the overall transparency of ICT products and services, ensuring security by default and design and addressing vulnerabilities more than what the ECCF has done.’

In summary, the ECCF’s EU added value is widely recognised by stakeholders. The framework enables harmonised certification, increases trust and transparency and streamlines processes across the EU, achieving outcomes that would be difficult for Member States to accomplish on their own.

6. Targeted consultation (interviews)

As part of the evaluation of ENISA and the ECCF, a structured interview programme was implemented to collect qualitative insights from key stakeholders. In total, 49 interviews were conducted for ENISA and 13 interviews for the ECCF, involving a diverse range of participants including ENISA staff and representatives of the Commission, national cybersecurity authorities, industry associations and international organisations. The interviews were designed to complement the survey and the call for evidence by providing detailed perspectives on the implementation of ENISA’s mandate, the functioning of the ECCF and the practical challenges and opportunities encountered by those directly involved in the European cybersecurity ecosystem.

ENISA

Interviewees consistently recognised ENISA’s changing and increasingly central role in the EU cybersecurity landscape. ENISA was widely regarded as a trusted coordinator and facilitator, particularly valued for its support to Member States, its expertise in policy implementation and its contribution to operational cooperation. However, resource constraints were frequently mentioned as a limiting factor, with stakeholders noting that ENISA’s expanding mandate and growing responsibilities were not always matched by adequate allocation or reallocation of financial and human resources. The agency’s matrix-based organisational model and its governance structure were viewed positively, but interviewees called for further strengthening of internal coordination and prioritisation mechanisms to manage periods of high workload and to ensure the efficient delivery of outputs. Stakeholders from national authorities and EU institutions emphasised ENISA’s valuable support in the implementation of cybersecurity policies, including the NIS Directive, the NIS2 Directive and sector-specific regulations. ENISA’s technical guidance, best practice-sharing and capacity-building initiatives were cited as particularly beneficial for smaller Member States and as being effective in harmonising approaches across the EU. However, some stakeholders expressed a need for more tailored, sector-specific guidance and for better dissemination of ENISA’s outputs to ensure they are taken up in practice by a wider range of stakeholders.

The interviews highlighted ENISA’s added value in fostering cooperation and knowledge-sharing among Member States, EU institutions and other stakeholders. The agency’s role as a facilitator of networks such as the CSIRTs network and EU-CyCLONe was also appreciated. Nonetheless, several interviewees identified areas for improvement, including the need for more structured and regular engagement with industry, SMEs and civil society, as well as greater transparency and inclusiveness in stakeholder consultations. Some stakeholders suggested that ENISA could further strengthen its engagement with international partners and standardisation bodies to augment the global relevance of EU cybersecurity policies and certification schemes.

Across all interviews, there was a clear call for greater clarity, flexibility and resourcing at both the EU and national levels. Stakeholders recommended beefing up ENISA’s mandate and resources to enable the agency to fulfil its expanding responsibilities, particularly in the areas of operational cooperation, policy support and certification scheme development. Suggestions included formalising ENISA’s leadership in key networks, improving internal coordination and increasing the frequency and depth of stakeholder engagement. Interviewees also emphasised the need for ongoing investment in capacity building, knowledge-sharing and the development of sector-specific guidance.

ECCF

Interviewees provided detailed feedback on the implementation and governance of the ECCF. While the framework was recognised as a valuable tool for harmonising cybersecurity certification across the EU, stakeholders identified several challenges. Delays in the adoption of certification schemes, particularly the EUCC and EUCS, were attributed to complex technical requirements, political sensitivities and coordination difficulties among Member States and EU institutions. Industry representatives and national authorities expressed concerns about insufficient involvement in scheme development, limited transparency in decision-making processes and the need for clearer roles and responsibilities among key actors, including ENISA, the Commission and the ECCG and SCCG groups.

Interviewees also highlighted the importance of aligning certification schemes with international standards and ensuring consistency with other EU legislative instruments, such as the NIS2 Directive and the proposed CRA. The voluntary nature of schemes was not seen as a major problem, but the lack of experience in developing certification schemes contributed to delays and uncertainty. Stakeholders called for more structured engagement in the framework through clear processes and realistic timelines, as well as for regular meetings with clear objectives and sufficient time to prepare.

There was a strong emphasis on the need for more effective mechanisms to resolve political and technical deadlocks during scheme development, as well as for greater transparency and information-sharing throughout the process. Interviewees stressed that the ECCF’s added value would only be fully realised if these governance and operational challenges were addressed, enabling the framework to deliver on its promise of a harmonised and effective European cybersecurity certification landscape.

7. SWOT and recommendations workshops

As part of the evaluation process for ENISA and the ECCF, two dedicated workshops were organised to facilitate collaborative analysis and gather targeted recommendations. These workshops brought together a wide range of stakeholders, including experts from academia and representatives from ENISA, DG CNECT, Member States and the study team. The workshops were conducted online and structured to maximise technical exchange, validation of findings and the formulation of actionable suggestions for future improvements.

7.1 SWOT Workshop

The SWOT workshop was held on 21 May 2024 via MS Teams, with 32 participants, including six study team members. The objective was to conduct an analysis of ENISA and the ECCF, using data collated by the study team and direct input from stakeholders. The workshop aimed to improve understanding of the strengths, weaknesses, opportunities and threats facing both ENISA and the ECCF and to foster dialogue on potential improvements to the Cybersecurity Act.

ENISA SWOT Results

Stakeholders confirmed the strengths identified by the study team, including ENISA’s positive reputation within the EU cybersecurity community, the quality of its publications, effective collaboration among Member States, harmonisation efforts and capacity building. Its weaknesses were also attested, including resource shortages, strategic rigidity, potential delays, lack of clarity of ENISA’s policy role and limited operational support. Opportunities identified included increased impact and services, collaborative training initiatives, the harmonisation of standards, policy engagement and an expanded mandate for more direct operational roles. Threats discussed included rapid technological advancement, the proliferation of funding initiatives, unexpected resource constraints, complacency and geopolitical instability.

ECCF SWOT Results

The ECCF SWOT analysis was similarly confirmed by participants. Strengths included boosting cooperation at EU level, adaptability to evolving technologies and legislative frameworks, the ability to mobilise experts and contribute to the development of standards, ENISA’s capacity to ensure smooth scheme adoption, its relevance for internal market objectives and the added value of EU action. Weaknesses raised were the time-consuming scheme adoption processes, complexity in scheme development, susceptibility to geopolitical pressure and policy priority changes, a perceived lack of involvement of industry and external stakeholders and an unproven capacity to support cybersecurity preparedness and transparency.

Opportunities identified included greater flexibility and usefulness in tackling emerging threats such as artificial intelligence, strengthening Member State capabilities, the possibility of using certification as a presumption of conformity, potential mandatory requirements from upcoming legislation, the extension of its scope to managed security services, and providing impetus to standardisation processes. Threats included a lack of resources, potential shifts in policy priorities, increasing geopolitical instability, the risk of overlaps with other EU legislation and a risk of internal market distortions due to national cybersecurity requirements.

7.2 Recommendations Workshop

The recommendations workshop took place on 12 July 2024, also via MS Teams, with 77 participants, including seven study team members. The aim was to gather ideas for improving the performance of ENISA and the ECCF, drawing on the evaluation results and stakeholder input.

ENISA Recommendations

Polling results showed an average score of 3.9 (on a scale from 1 (not at all) to 5 (completely/fully agree)), indicating general agreement with the evaluation findings but with some reservations, particularly regarding efficiency and internal governance. Stakeholders emphasised the need for adequate financial resources and stressed the importance of maintaining a clear focus within the mandate to avoid resource dilution. Coordination with other bodies such as ECCC, EU-CyCLONe and CSIRT was highlighted as a way of avoiding duplication and improving efficiency.

Suggestions for governance included creating platforms for Member States to share information, streamlining the production of reports and balancing ENISA’s advisory and executive roles. The importance of seconded members from Member States for increasing alignment with national priorities was noted, though retaining qualified staff remains a challenge. Stakeholder engagement should be efficient and tailored to different audiences, with feedback systematically incorporated into future planning.

ENISA’s position in the EU cybersecurity landscape could be strengthened by it prioritising services and support to Member States, acting as an impartial facilitator and fostering robust cooperation with bodies such as the ECCC. Suggestions were also made for ENISA to establish cooperation with US entities such as MITRE 17 and NIST (regarding the NVD) 18 to improve threat intelligence and vulnerability management.

ECCF Recommendations

Polling results for the ECCF averaged 3.8 (on a scale from 1 to 5), indicating general agreement with the evaluation findings, but revealed more disparate views on coherence and added value. Discussions focused on the future purpose and scope of EU certification, with participants divided on whether the ECCF should address non-technical threats. Some argued for keeping the framework technical while others saw value in using certification to address strategic issues.

Process improvements were suggested across all stages of scheme development, including early preliminary assessments, realistic timeframes, increased involvement of the ECCG and early legal advice from the Commission. Greater independence for ENISA in scheme development and limiting political elements were also recommended. Simplifying administrative processes and considering the elimination of implementing acts for voluntary schemes were proposed to streamline adoption. Participants discussed the future roles of stakeholders, including the role of the ECCG in the maintenance of the schemes. Alignment with existing standardisation efforts was emphasised, as was the need for more industry involvement and input from independent experts.

(1)

Commission Staff Working Document Impact Assessment (2017) accompanying the proposal for a Regulation of the European Parliament and of the Council on ENISA, the ‘EU Cybersecurity Agency’ and repealing Regulation (EU) 526/2013 and on information and communication technology cybersecurity certification (‘Cybersecurity Act’).

(2) Regulation (EU) No 526/2013 of the European Parliament and of the Council of 21 May 2013 concerning the European Union Agency for Network and Information Security (ENISA) and repealing Regulation (EC) No 460/2004 (Text with EEA relevance) ( http://data.europa.eu/eli/reg/2013/526/oj ).

(3) Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union, amending Regulation (EU) No 910/2014 and Directive (EU) 2018/1972, and repealing Directive (EU) 2016/1148 (NIS 2 Directive) (Text with EEA relevance) ( http://data.europa.eu/eli/dir/2022/2555/oj ).

(4)   EUR-Lex - 52017SC0500 - EN - EUR-Lex

(5)

 McAfee & Center for Strategic and International Studies, ‘Net Losses: Estimating the Global Cost of Cybercrime’, 2014.

(6)

As per CSA 2017 IA; Source: White House, Factsheet Cybersecurity National Action Plan.

(7)

CSA 2017 IA.

(8) Contractual Public-Private Partnership on cybersecurity, signed by the European Commission and the European Cyber Security Organisation (ECSO) on 5 July 2016.

(9)

Commission Staff Working Document: Report on the public consultation and other consultation activities of the European Commission for the preparation of the EU Cybersecurity contractual Public-Private Partnership and Accompanying Measures, accompanying the document Commission Decision on the signing of a contractual arrangement on a public-private partnership for cybersecurity industrial research and innovation between the European Union, represented by the Commission, and the stakeholder organisation, SWD(2016) 215 final.

(10) In this 2016 public consultation, a section was devoted to the topic of ICT security certification. 240 stakeholders from national public administrations, large businesses, SMEs, microbusinesses and research bodies responded to the section on certification.

(11) JOIN(2020) 18 final.

(12) PPMI, Intellera Consulting and PwC (2024): Study to support the evaluation of the European Union Agency for Cybersecurity (ENISA) and the European Cybersecurity Certification Framework – Final Report.

(13)  NIS Investments Report 2020, https://www.enisa.europa.eu/publications/nis-investments

(14)  ENISA, Annual Activity Report, 2020.

(15)  ENISA, Annual Activity Report, 2023.

(16)

Other EU and national bodies working on cybersecurity and digital privacy include various Commission directorates, the European External Action Service and other EU bodies and agencies such as BEREC, EUROPOL and CERT-EU, national cybersecurity competent authorities or regulators, national CSIRTs/Computer Emergency Response Teams and more recently the European Cybersecurity Competence Centre (ECCC) and its network of national coordination centres (NCCs).

(17)  https://www.mitre.org/

(18)   NVD - Home