EUROPEAN COMMISSION

Brussels, 6.4.2022

SWD(2022) 103 final

COMMISSION STAFF WORKING DOCUMENT

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Evaluation of the European Union Agency for Law Enforcement Training

based on Article 32 of the CEPOL Regulation (EU) 2015/2219 of the European Parliament and of the Council of 25 November 2015 on the European Union Agency for Law Enforcement Training (CEPOL) and replacing and repealing Council Decision 2005/681/JHA

{COM(2022) 153 final}

Table of contents

1.Introduction

2.Background to the intervention

3.State of Play

4.Method

5.Analysis and answers to the evaluation questions

6.Conclusions

List of Abbreviations

Member State Abbreviations

Other

1.    Introduction

The European Union Agency for Law Enforcement Training (CEPOL) was founded in 2000 and its current legal mandate entered into force on 1 July 2016, resulting from Regulation (EU) 2015/2219 (hereinafter: the CEPOL Regulation). As per Article 3 of the Regulation, CEPOL’s mandate includes supporting, developing, implementing and coordinating training for law enforcement officials. The external study supporting this evaluation was conducted by a consortium of RAND Europe and Ernst & Young (EY).

Purpose and scope

As foreseen in Article 32 of the CEPOL Regulation and based on the Terms of Reference for the external evaluation, the contractors assessed in the evaluation study:

1.    The effectiveness and efficiency, relevance, coherence and EU added value of CEPOL and of its working practices since 2015.

2.    Lessons learned and recommendations to better respond to the challenges posed by the constantly changing environment, against the background of the current authorised financial and human resources for CEPOL.

The evaluation also had to verify to what extent the provisions establishing the Agency are aligned with the Common Approach on Decentralised Agencies, looking in particular into the Agency's governance, internal structures and procedures and applicable administrative and financial rules. Another intention of the evaluation was to provide solid grounds for any possible decision concerning the Agency’s future mandate.

The results of the evaluation are expected to be used by CEPOL, its Management Board, the Commission, the Council, and the European Parliament and other Justice and Home Affairs agencies in the future decision-making process regarding CEPOL’s functions and operations.

The scope of the evaluation in terms of subject, timeframe, stakeholders consulted, and territory was:

•    Subject: The evaluation addresses the objectives of the Agency as established in Article 3 of the CEPOL Regulation 2015/2219 of 25 November 2015. In doing so, it covers: the working practices of CEPOL; the provisions establishing CEPOL; CEPOL’s governance, internal structures, procedures and administrative and financial rules; CEPOL’s internal (within the EU) and external (with third countries) activities; and how the Agency cooperates with other EU agencies and international organisations.

•    Timeframe: The evaluation covers the period 2015 to 2020.

•    Stakeholders: The following categories of stakeholders have been consulted for the evaluation: CEPOL staff; the CEPOL Management Board; CEPOL National Units (CNUs) and National Contact Points (NCPs); EU institutions; Justice and Home Affairs (JHA) agencies; European level organisations; International organisations; third countries; Framework Partners (FWPs); Training participants; and the general public. A full list of stakeholders consulted for the evaluation is provided in Annex II.

•    Territory: The evaluation covers CEPOL’s activities with all EU Member States (except Denmark ), third countries with which CEPOL has working arrangements, cooperation agreements and similar arrangements, and international organisations and EU agencies.

2.    Background to the intervention

Established as European Police College by Council Decision 2000/820/JHA, CEPOL became operational on 1st January 2001 and later an Agency in 2005. The revised CEPOL Regulation 2015/2219 of 25 November 2015 entered into application on 1st July 2016. As of that date, CEPOL became the European Union Agency for Law Enforcement Training (the acronym remained unchanged).

The general objective of the new Regulation is to align CEPOL with the Lisbon Treaty and to give it the appropriate legal mandate to implement the Commission Communication on “Establishing a Law Enforcement Training Scheme” (LETS) 1 , as well as to improve the governance of CEPOL by seeking increased efficiency.

The new legal framework has modified CEPOL’s mandate. It now includes a wider target group (all law enforcement agents as opposed to only senior police officers), a more active role in the delivery of training activities (the Agency only used to coordinate national academies), the task to identify EU strategic training needs, and a more prominent activity in the external dimension.

CEPOL's mission is to provide training and learning opportunities to law enforcement officers on issues vital to the European police cooperation. Training covers topics ranging from leadership to law enforcement techniques and from EU cooperation to economic crime. Activities are designed to facilitate the sharing of knowledge and best practices and to contribute to the development of a common European law enforcement culture.

CEPOL contributes to a safer Europe by facilitating cooperation and knowledge sharing among law enforcement officials of the EU Member States and to some extent, from third countries, on issues stemming from EU priorities in the field of security, notably from the EU Policy Cycle on serious and organised crime, now commonly know as EMPACT (European Multidisciplinary Platform Against Criminal Threats) 2 . Moreover, CEPOL assesses training needs to address EU security priorities. The agency’s annual work programme is built with input from its network and all concerned stakeholders, resulting in topical and focused activities designed to meet the needs of Member States in the priority areas of the EU internal security strategy. The Agency also supports the EU Neighbourhood Partners through a few capacity building projects, which cover in different formations the Eastern Partnership Region, the MENA countries, Turkey and the Western Balkans.

CEPOL’s headquarters were relocated to Budapest (Hungary) as of 1st October 2014, at the initiative of the Member States, which rejected the European Commission's proposal of merging CEPOL and Europol in The Hague. The relocation from London to Budapest happened despite the European Commission's negative opinion.

Description of the intervention and its objectives

In accordance with the Better Regulation guidelines for evaluations of the European Commission 3 , the evaluation examines the effectiveness, efficiency, relevance and coherence of the actions of CEPOL, as well as at the achieved EU added value of these actions. Table 1 below sets out the definition of each of these components.

Table 1: Overview of cross-cutting evaluation components

Source: Based on Tool #42: Identifying the evaluation criteria and questions 4 .

Intervention logic

To aid the analysis, an intervention logic was created for this evaluation (see Annex IV) 5 . This is the main tool to understand causal and logical links among the objectives of the Agency and the relevance, coherence and effects expected from work and activities.

The intervention logic highlights CEPOL’s overarching objective, which is to support, develop, implement, and coordinate training for law enforcement officials. This is a crucial role that is intended to ensure law enforcement officials are equipped to protect human rights, prevent and fight serious crime and terrorism, and maintain public order. The specific objectives of CEPOL are to:

·Support Member States in the provision of training to law enforcement officials, in particular on elements of cross-border and European law enforcement cooperation;

·Support Member States in developing bilateral and regional cooperation through the delivery of multilateral training;

·Develop, implement and coordinate training on specific thematic areas;

·Develop, implement and coordinate the training of law enforcement officials when preparing for participation in EU missions and in capacity building activities in non-EU countries;

·Provide training to law enforcement trainers and supporting the development and exchange of best practice;

·Develop, upgrade and evaluate learning tools and methodologies that support the learning and development of law enforcement officials. 6

Evaluation framework

The methodological approach was derived from a careful analysis of the 33 evaluation questions set out in the terms of reference and presented in Annex V, and the production of evaluation grids separated by each of the five evaluation criteria. Building on the evaluation questions, the evaluation grids include:

·Judgment criteria: statements that need to be confirmed or not confirmed by the analysis.

·Analytical approach to be used to answer the evaluation question. The type of analysis proposed informed the type of information collected.

·Indicators: quantitative and qualitative measures that support the analysis.

·Data sources: quantitative and qualitative sources to feed the analysis and indicators.

In this report, the evaluation criteria are addressed in the order of the Better Regulation Guidelines. The evaluation questions are provided in Annex V.

Baseline and points of comparison

This is the first evaluation carried out by the Commission; the previous evaluation was commissioned by CEPOL and covered the preceding five years.

3.    State of Play

Description of the current situation

The new legal basis resulting from the 2015 Regulation (EU) 2015/2219 established CEPOL as a European Agency. The Regulation was implemented following the European Parliament’s request for the development of a strong EU framework for judicial and police training and built on the work of the Stockholm Programme. 7 The Regulation outlined a range of new tasks that included capacity building in third countries, promoting the mutual recognition of training between Member States and the assessment of EU initiatives. Moreover, the new legal basis broadened CEPOL’s remit in terms of external relations and target audience 8 , and led to changes in its governance structure. The current objectives as outlined in the 2015 Regulation (Article 3) are supporting, developing, implementing and coordinating training for law enforcement officials. More specifically:

ØSupport Member States in the provision of training to law enforcement officials, in particular on elements of cross-border and European-level policing.

ØSupport Member States in developing bilateral and regional cooperation through the delivery of multi-lateral training.

ØDevelop, implement and coordinate training on specific thematic areas.

ØDevelop, implement and coordinate the training of law enforcement officials when preparing for participation in EU missions and in capacity building activity in non-EU countries.

ØProvide training to law enforcement trainers and support the development and exchange of best practice.

ØDevelop, upgrade and evaluate learning tools and methodologies that support the learning and development of law enforcement officials 9 .

CEPOL aligns its activities with relevant EU strategies and policies, such as the European Agenda on Security, focusing its learning and training outputs on threat areas such as terrorism, serious and organised crime, cybercrime and the smuggling of migrants 10 . CEPOL has identified cybercrime as the key priority for the coming years, demonstrated by the launch in 2019 of the CEPOL Cybercrime Academy (CCA). These strategic areas also address the priorities outlined in the EMPACT 11 .

Another key task of CEPOL is to conduct the EU Strategic Training Needs Assessment (EU-STNA). This task emerged from an acknowledgment in 2013, following the establishment of the Law Enforcement Training Scheme (LETS), of the need to address the lack of a systematic process for identifying the evolving training needs for law enforcement in the EU. The Regulation identified that this was necessary to avoid duplication and ensure better coordination of training activities. 12 The first EU-STNA was launched in 2017. This led to a consolidated list of 184 EU-level training needs across 21 thematic categories that were distributed among the Member States, with a request to prioritise and rank the training needs. CEPOL prepared a report based on these findings, which was later endorsed by the European Parliament. In addition to the EU-STNA, CEPOL also undertakes an Operational Training Needs Analyses (OTNA) on the key priority topics outlined by the EU-STNA. The aim of this exercise is to gain a detailed understanding of the profile of the officials to be trained, as well as the proficiency and urgency of the training to be delivered. In 2020, OTNAs were carried out in five areas: illegal immigration; trafficking in human beings; criminal finances; money laundering and asset recovery; and drug trafficking. Multiannual training portfolios are informed by the outcome of OTNAs.

CEPOL is governed through a Management Board, made up of representatives from 26 Member States and the European Commission. The Chair of the Management Board is a representative of one of the three Member States that have jointly prepared the Council of the European Union’s 18-month programme. The board meets twice per year and works to adopt CEPOL’s annual and multi-annual work programming and its annual budget. Programming documents set out a number of ‘goals’ along with key performance indicators (KPIs) used to monitor future progress in consultation with the Member States. The Management Board also provides suggestions to improve CEPOL’s internal processes and compiles the Consolidated Annual Activity Report (CAAR) for each financial year, which assesses the delivery of activities against requisite work programmes and KPIs.

CEPOL is headed by an Executive Director who is accountable to the Management Board. The responsibilities of the Executive Director include, but are not limited to, the day-to-day administration of CEPOL; making proposals to the Management Board as regards the establishment of CEPOL's internal structures; implementing decisions adopted by the Management Board and preparing the draft multi-annual programming and annual work programmes and submitting them to the Management Board. Between 2016 and 2018 CEPOL’s operations services were reorganised, with a dedicated International Cooperation Unit (ICU) now in place. In addition, e-learning provision and research and analysis sectors were added to an updated Training and Research Unit and legal functions became incorporated into the Director’s Office, rather than under corporate services. The Director’s office also combines external relations, communications, and quality management functions, among others (see Figure 1 below). CEPOL has dedicated CNUs in every Member State to provide information and assistance to law enforcement officials wanting to participate in CEPOL’s activities. CNUs also support CEPOL’s operations and influence the annual work programming.

Figure 1: CEPOL’s Management and Governance Structure 13  

Source: CEPOL Single Programming Document 2020-2022.

The aforementioned CEPOL Management Board oversees the Agency’s governance, risk management and internal control practices. This process takes place through the use of tools such as progress reports (including performance indicators and audit recommendations), annual activity reports and internal/external audits.

CEPOL’s internal organisational structure is composed of two departments, which are coordinated by the Head of Operations and Head of Corporate Services, as well as three units under the supervision of a head of unit (Head of Training and Research Unit, Head of International Cooperation, Head of Finance, Procurement and Travel). At the level of units, all managers are accountable for the achievement of objectives and operational and internal control and comply with the reporting arrangements within the Agency. The main responsibility of the CEPOL units is to manage risk and controls on a daily basis. 14  CEPOL also uses an Internal Control Framework (ICF) to support its decision making to achieve objectives and sustain operational and financial performance, respecting rules and regulations. CEPOL does not have a dedicated managerial position in charge of risk management and internal control; this is a shared management responsibility. In addition to the ICF, CEPOL has followed the Quality Management System Standard ISO 9001:2015, since February 2017, when the certification was obtained.

Staffing

In 2020, CEPOL’s staff numbered 84 in total (see Table 2). According to CEPOL’s 2020 Establishment Plan, the Agency aimed to have 33 Temporary Agent (TA) positions in 2020, complimented by 19 Contract Agent (CA) positions and 7 Seconded National Expert (SNE) positions. Fluctuations in staffing levels can be attributed to varying needs for the delivery of EU projects funded from grant agreements, such as the Western Balkan Financial Investigations project concluded in March 2020.

Table2: Number of CEPOL staff, 2015 – 2020

Source: CEPOL Consolidated Annual Activity Reports (CAAR) (2015 – 2020). Notes: Staff numbers are those at the end of each year.

The most recent CAAR, published in 2021, noted that the staff turnover at CEPOL continued to be relatively high. Consequently, the Agency used a number of interim staff and SNEs to fill in for staff absences, until recruitment of statutory staff was completed and continued the reclassification of staff and functions, as well as maintaining teleworking and flexitime arrangements.

Budget and finances

CEPOL’s activities are funded through an annual subsidy from the European Commission, as well as grant agreements specifically for activities delivered through CEPOL’s ICU 15 . As shown in the table 3 below, CEPOL’s overall revenue increased gradually over the period 2015 to 2020, mainly due to a steady increase in the revenue from grant agreements. There was no significant increase in CEPOL’s EU annual subsidy in the period before and after the new legal basis came into effect. That is, in 2012, 2013 and 2014 the EU annual subsidy was €8.45 million, €8.45 million, and €8.35 million respectively. At the time of the new legal basis, in 2015 and 2016, the revenue from the EU subsidy remained stable. The subsidy increased to over €10 million in 2018, decreased again to €9 million in 2019 and increased again to over €10 million in 2020.

Table 3: CEPOL’s annual budget (‘000 Euro)

Source: CEPOL’s CAARs (2015 - 2020).

CEPOL’s activities are organised as part of its annual work programme (AWP), which is established at the outset of each year and agreed upon with the Member States and the Commission. The programme contains detailed objectives, expected results and performance indicators. For example, CEPOL routinely monitors the number of participants for each activity and measures how this has changed from year to year, against its annual targets. The AWP also features a description of the actions to be financed and an indication of the financial and human resources allocated to each action, in accordance with the principles of activity-based budgeting and management.

CEPOL offers a range of activities for the learning and development of law enforcement officials in the Member States and third countries. Table 4 provides data on the number of training activities delivered by the Agency over the period 2015 to 2020 by type of activity and year. While CEPOL does itself deliver some activities, including residential and online learning, CEPOL implements most of its activities through the allocation of grants to Framework Partners identified through restricted Call for Proposals that are published every four years.

Table 4: Number of training activities by type and year 16

Source: CEPOL CAARs (2015 - 2020).

Residential training activities involve participants from across the EU coming together in one location (often not at CEPOL’s headquarters in Hungary) for several days of knowledge, expertise and skill transference. The Activity Organiser is responsible for preparing the content, budget planning, implementation and logistics.

The CEPOL Exchange Programme (CEP), formerly the European Police Exchange Programme (EPEP) allows law enforcement officers to spend a week in another country with a counterpart. The CEP, which is modelled on the Erasmus programme, offers participants an exchange of knowledge and good practice, with the opportunity to network and form long lasting professional relationships.

All of CEPOL’s activities are supported by the Agency’s online training platform, Law Enforcement Education (LEEd), which is open to all EU law enforcement officers and is free to use. LEEd replaced CEPOL’s old e-learning platform, ‘e-Net’, in 2020. CEPOL’s online learning offer comprises webinars, online courses and online modules.

The CEPOL European Joint Master Programme (EJMP) was an EU academic programme developed in 2015 with the objective of addressing common challenges of police cooperation in the area of internal security. The EJMP targeted senior law enforcement officers or experts in general management and specialist positions. From 2020 the continuation of the EJMP was put on hold after the Internal Audit Service identified non-compliant practices around the setup of the programme.

A more recent activity was the 2019 inauguration of the Cybercrime Academy (CCA). Built to train up to 100 participants simultaneously, the establishment of the academy reflects CEPOL’s prioritisation of cybercrime as a growing threat to European economies and societies. Until April 2021, the CCA had delivered 38 training activities.

In 2017 two CEPOL Centres of Knowledge (CKCs) were established in the areas of counterterrorism and CSDP Missions to pilot a new business model of implementing courses through multiannual direct awards granted to Framework Partners. CEPOL’s CKC model intended to bring together subject experts from Member States and relevant EU Agencies to design multi-annual training portfolios (including residential and online activities) on given topic areas. CEPOL would be responsible for designing and implementing courses within the CKC, in collaboration with hosting training institutions in Member States. The pilot aimed to increase efficiency by encouraging Framework Partners’ ownership of projects, reducing the administrative burden on applicants generated by the annual grants’ procedure as well as addressing a consistent underspending of resources and limited capacity of Framework Partners to implement CEPOL activities. Further to an external evaluation, CEPOL designed and implemented a hybrid business model in 2020, with a clearer division of responsibilities. In this hybrid model, CEPOL began to enlist the support of CKCs, composed of representatives from Member States, as well as from relevant EU JHA agencies and bodies (e.g. Europol, Frontex, EASO), with expertise on a specific topic listed in the call for experts. The CKC groups are tasked with helping CEPOL to prepare its multiannual training portfolio for each thematic area, based on strategic and operational training needs assessments. In 2020, one new CKC was piloted in line with this new business model, with a thematic focus on counterterrorism, and one additional pilot is being set up in 2021 focusing on law enforcement cooperation, information exchange and interoperability.

One of CEPOL’s multiannual objectives is to contribute to and encourage the development of research relevant for training activities and to disseminate research findings. To fulfil this objective, CEPOL aims to integrate developments in research and promote the creation of stronger partnerships between universities and law enforcement training institutes in the Member States. In terms of research outputs, since 2009 CEPOL has published several bulletins on an annual basis on its website. From 2018 onwards, these bulletins were published as the ‘European Law Enforcement Research Bulletin’. There were on average two research bulletins published each year between 2015 and 2020.

Described as the Agency’s ‘flagship event’ the CEPOL Annual Research and Science Conference aims to provide insight into the future challenges of modern European policing, although no conference has been held in 2017-20. The conference celebrated its 15th anniversary in 2016 and brought together researchers and practitioners from across the EU to explore the challenges of law enforcement training and education 17 . In 2017 the conference was organised in collaboration with the Hungarian National University of Public Services and focused on ‘Innovations in Law Enforcement – implications for practice, education and civil society. CEPOL is also an occasional host of workshops. Additionally, CEPOL has an e-library and registered users of CEPOL’s LEEd platform have online access to more than 300 European and international titles of e-journals covering areas such as police science, law enforcement and criminal justice.

CEPOL is required by its mandate to design and implement its activities in accordance with the challenges and needs of the EU. The capacity building in third countries has worked towards main policy documents, including the European Agenda on Migration, the European Agenda on Security and the Council Conclusions on EU External Action on Counter-terrorism. The EU/Middle East and North Africa (henceforth ‘MENA’), the Counter-Terrorism Training Partnership (1 and 2) and Financial Investigation in-service training Western Balkans are projects, which have aimed to enhance the Agency’s role in strengthening the internal security of the immediate neighbourhood of the EU. The activities in this area include residential stays, regional workshops, evaluation visits and round tables.

4.    Method

Short description of methodology

Limitations and robustness of findings

There are a number of limitations of the data collected to inform the evaluation, which should be borne in mind when interpreting the findings. Below we summarise the limitations and mitigation measures taken to address them.

1.Measuring effectiveness: Measurement of the Agency’s effectiveness is limited by lack of statistical, financial, and administrative data to test, challenge and validate the expert judgements and stakeholder opinions collected through fieldwork activities. For example, there is an absence of data to assess whether the skills and knowledge of CEPOL training participants have actually increased as a result of the training provided.

2.Relying on self-reported data: Data collected from stakeholders through interviews and surveys is self-reported, may be subject to recall bias and may reflect the specific agenda of those consulted.

Mitigation measure to points 1 and 2: Triangulation of self-reported data with data analysed from documentary sources including factual information on the functioning of CEPOL, information on CEPOL’s activities, previous evaluation documents, and documents related to CEPOL’s partnerships.

3.Attributing outcomes: In the absence of a counterfactual, there are challenges in unequivocally attributing outcomes to the activities of CEPOL. The aim of this evaluation was to assess the strength of evidence for CEPOL’s progress against the intended intervention logic, rather than assess direct, verifiable causality based on strong empirical evidence.

Mitigation measure: The evaluation is a contribution analysis, rather than an impact evaluation assessing causality.

4.Quantifying benefits: The inability to quantify the benefits of CEPOL means that this analysis is unable to make any concrete conclusions about the cost-effectiveness of the Agency.

Mitigation measure: The efficiency analysis focuses on the costs of CEPOL’s activities and the efficiency of the Agency in light of the resources available.

5.Representativeness of stakeholder views: There may be selection bias because those interviewed and surveyed were selected by the research team following a stakeholder mapping exercise (involving the Commission and CEPOL) and may not be representative of all stakeholders of CEPOL.

Mitigation measure: Selection of stakeholders across a range of groups including those at the EU, Member State, and international levels.

5.    Analysis and answers to the evaluation questions

This section includes a summary of the analysis carried out during the evaluation.

5.1 Effectiveness

Following the Better Regulation Guidelines, this criterion examines why an objective has or has not been met, as well as factors driving or hindering progress. The evaluation looked at the effectiveness in relation to the general and specific objectives outlined in the intervention logic (Annex IV). Given the diverse potential beneficiaries of CEPOL’s work, the assessment of effectiveness attempts to provide sufficient details so as to explore what is effective, when and for whom.

The analysis of available evidence shows that CEPOL’s AWPs reflect both the Agency’s objectives and relevant EU priorities. Most CNUs/NCPs indicated that, in their opinion, almost all of CEPOL’s objectives have been reflected ‘to a great extent’ in the Agency’s actions and activities over the past five.

Multiple interviewees noted that CEPOL struggles to address the enlarged target audience introduced by the 2015 Regulation. Compared to the total number of law enforcement personnel in the EU, CEPOL directly reaches only a minimal proportion of its target audience. It should be noted that CEPOL is not expected to reach a specific numeric target of law enforcement practitioners by mandate, as the Stockholm Programme tried to set a quantifiable target for law enforcement training activities, yet the proposition did neither find its way in the LETS Communication, nor in Regulation 2015/2219. Given CEPOL’s relatively small size (69 staff members) and annual budget, it is difficult for CEPOL to directly reach a sizeable proportion of the target audience. Police officers alone in the 26 Member States covered by CEPOL amount to around 1.5 officers 18 , a figure that does not account for the extended targeted audience introduced by the 2015 Regulation 19 . CEPOL activities in 2019, its best year on record to date in terms of outreach, reached a total of 34,615 participations ( Table 5 ). The available data 20 show that CEPOL training participants usually take part in more than one activity, and with the current system CEPOL is unable to distinguish between unique participants (whereby individuals would be counted only once, even if they attended multiple CEPOL courses) and participations (total attendance to CEPOL activities, counting individuals multiple times for each activity they attended), leading to an inaccurate representation of its outreach activities in the Annual Reports. Even admitting that CEPOL managed to substantially enlarge its audience, CEPOL must therefore rely on indirect means to extend its reach and address the broadest target audience possible, most notably through the ‘cascading mechanism’ and its digitalisation strategy.

Table 5: Number of participants by training type and CEPOL outreach capacity assuming it only targeted police officers

Source: CEPOL CAARs (2015-2020) (participants), Eurostat (police officers).

Given the above, CEPOL should aim to broaden the reach of its target audience through the cascading mechanism resulting from the training of trainers and peer-to-peer transfer of knowledge. The evaluation found that the extent of cascading is inconsistent across Member States and largely out of CEPOL’s control. As a matter of fact, Regulation 2015/2219 does not include the explicit promotion of the peer-to-peer transfer of knowledge in the remit of CEPOL, although Article 3 includes the objective to “train trainers and assist in improving and exchanging best learning practices” and cascading seems the logical means to ensure a broader outreach, given the Agency’s current capacity. Information collected shows that CEPOL, while not actively promoting cascading, recognises its importance and attempts to capture its impact, although with mixed results. CEPOL uses its customised version of Kirkpatrick’s methodology for training evaluation to collect information from training participants on the extent to which they transfer knowledge acquired through CEPOL. However, CEPOL’s reporting on cascading is inconsistent – using slightly different metrics in different CAARs, precluding meaningful comparison (see Table 6).

Table 6: Accounts of the cascading effect reported by CEPOL

Sources: CEPOL CAARs (2015-2020).

Nonetheless, most of the surveyed CNUs/NCPs and training participants affirm that the knowledge and skills acquired through CEPOL’s training programmes are being transferred by training participants to their peers. Survey respondents were also of the view that cascading is not in CEPOL’s hands, however, some offered suggestions as to how CEPOL could better support cascading by: (i) actively including the requirement to envisage cascading activities for the participants as part of the training developed by CEPOL and by its FWPs, and to report on    those; (ii) developing a process to collect data to evidence the cascading; (iii) developing operational guidance on how cascading could be ensured at the national level; and (iv) actively reminding participants of the importance of cascading their knowledge to their peers; and (v) sharing best practices on cascading. All information collected by stakeholders point to the conclusion that capturing the impact of the cascading effect is indeed challenging, however, given (i) the small number of EU law enforcement officials trained per year as compared to the entire population; and (ii) the recognised importance attributed by CEPOL to cascading for multiplying the impact of training activities, there is a strong argument that CEPOL should do more to actively promote cascading.

CEPOL appears committed to fulfil its mandate when it comes to cross-border and European-level policing and invests its resources accordingly, with appreciable results. Regulation 2015/2219 details three specific aspects of cross-border cooperation and EU-level policing that CEPOL’s activities should focus on: (1) training on the implementation and use of international and Union instruments for law enforcement cooperation; (2) training for Union bodies and JHA agencies; (3) training on judicial cooperation and practical use of information exchange channels. The evidence collected shows that all three aspects are addressed, as CEPOL dedicates almost one-third of the total number of its yearly training activities to police cooperation and information exchange (see Table 7).

Table 7: Number of CEPOL activities dedicated to cross-border cooperation and EU-level policing between 2016 and 2019

Sources: CEPOL CAARs (2016-2020).

‘Strand 2’ of the LETS Communication indicated the need to raise awareness and to reinforce the use of law enforcement cooperation mechanisms and instruments to promote effective bilateral and multilateral cooperation among Member States. The analysis of SPDs and CAARs found that CEPOL covers a wide range of cooperation mechanisms and instruments through dedicated residential and online training activities. Furthermore, CEPOL exported these activities beyond the Union’s border and introduced collaborative mechanisms (such as the courses on JITs delivered in collaboration with the EJTN) to EU neighbours in the Western Balkans, effectively extending its support to bilateral and regional cooperation in law enforcement to the EU neighbourhood.

To ensure that the appropriate thematic areas are covered in its training offer, CEPOL developed and launched the EU-STNA to identify the evolving training needs of CEPOL’s target audience. The evaluation found that the introduction of the EU-STNA has strongly improved the prioritisation of training needs. Consistent with this, the overwhelming opinion across all consulted stakeholder groups is that CEPOL provides great added value with regard to training on specific crime areas compared to what is offered at the national level. Some of the interviewees indicated that CEPOL’s training offer underserved certain thematic areas and categories (e.g. forensics, customs, judiciary), and that the results of the EU-STNA overemphasise the priorities set at EU level by the EMPACT at the cost of priorities indicated by stakeholders at the national level.

CEPOL progressively increased the number of activities dedicated to EMPACT priorities - except in the year of COVID - and their relative weight within CEPOL activities increased accordingly ( Table 8). The interest from training participants in EMPACT priorities has grown accordingly, registering a four-fold increase between 2015 and 2019.

Table 8: EMPACT priorities: activities and participation over 2015-2020

Sources: CEPOL CAARS (2015-2019).

The number of CSDP-related activities has increased over time (see Table 9), although it seems to have reached the limits of CEPOL’s capacity evidenced by the fact that CSDP-related training ranks last among the 21 priorities identified by the EU-STNA, and most of CEPOL’s capacity is therefore allocated to top ranking priorities. One interviewed NCP confirmed that the interest in CSDP-related activities within CEPOL’s Management Board was low. As it emerged in case study 2 34 , CEPOL can leverage a comparative advantage in the provision of highly specialised training for civilian CSDP missions, but probably should leave generic pre-deployment training to other providers, as there is little added value to gain from CEPOL’s involvement.

Table 9: Quantitative indicators on CEPOL's CSDP related training

Sources: CEPOL CAARs (2015-2020).

Unlike CSPD-related training, CEPOL started providing capacity-building activities in third countries immediately following the introduction of the new legal basis. Regulation 2015/2219 enabled CEPOL to receive direct grants from the European Commission to implement projects in the EU neighbourhood. CEPOL has progressively received more resources through direct grants from the European Commission to expand its activity in third countries in the Western Balkans, in the Southern Neighbourhood and, most recently, to the Eastern Partnership countries (see Table 4 ). Representatives from partner third countries are particularly satisfied with the service-oriented approach of CEPOL, which distinguishes the Agency from other training providers in a crowded market (also see case study 2 35 ).

Table 10: Quantitative indicators on CEPOL's activities in third countries

Sources: CEPOL CAARs (2015-2020).

Training of law enforcement trainers was intended by the LETS Communication as a way to reduce training gaps across the EU, attributed in part to a lack of qualified trainers. CEPOL responded by developing residential and online activities dedicated to law enforcement trainers and training methodologies (see Table 1 1 ). The majority of surveyed stakeholders found that CEPOL achieved the objective of providing training to law enforcement trainers. More relevant to the stated objective of the LETS is the contribution that CEPOL provides in closing the gap between Member States. The majority of survey respondents within all stakeholder categories believe that CEPOL provides great added value compared to training for trainers provided at the national level. It is worth noting that in interviews ‘training the trainers’ emerged as a key enabler of the cascading of knowledge from CEPOL’s activities to Member States. Yet, case study 3 36 found that CEPOL’s train the trainers offer is comparatively smaller than that of other EU JHA agencies. Consideration should be given to expanding it.

Table 11: Number of training activities dedicated to train the trainers

Source: CEPOL training catalogues (2015-2020). *In 2020, two residential courses were planned but not implemented due to Covid-19 restrictions.

Knowledge-sharing and exchange of best practices, as seen across the CAARs of 2015-2020, was pursued through the creation of a cluster of services to develop CEPOL “into a European law enforcement knowledge base”. As described above, these services include CEPOL’s e-Library, the Lecturers, Trainers and Researchers Database (LTRdb), the publication of the Bulletin (CEPOL’s own scientific periodical), and the Research and Science Conference. The majority of stakeholders from all surveyed categories believed that CEPOL ‘fully achieved’ the objective of supporting the development and exchange of best practice.

The review of CEPOL’s AWPs between 2015 and 2020 showed that CEPOL actively seeks to develop, evaluate and upgrade its training digital delivery tools. CEPOL’s objective to upgrade and evaluate new learning methodologies has been addressed both in-house by CEPOL’s Training and Research Unit, and through the network of national training providers. The results are different ‘blended training’ methodologies, which entail a mix of training tools, including event workshops, residential training modules and coaching support, with different degrees of interactivity. According to interviewees, there is not one good way of blending the different training tools, and results depend on how individual participants respond to different kinds of training. Yet, from the evaluation it emerged that law enforcement officials respond better to practical cases and operational training, including elements such as tabletop exercises and simulations, suggesting that CEPOL could further tailor its training methodologies to its target audience.

Throughout the 2015-2020 period, CEPOL reached and exceeded most of its targets and it actively revised the set of Key Performance Indicators (KPIs) to include new challenges and address emerging issues (see table 45 at page 134 of the Final Report in the external Evaluation for the full list of KPIs). The few exceptions align with issues identified by this evaluation and presented in this report. CEPOL occasionally missed targets on: (i) reducing the flow of statutory staff turnover, (ii) meeting its budgetary commitments/payments, and (iii) the implementation of activities under Framework Agreements.

Overall, the review of the number of activities (see Table 1 2 ) implemented by CEPOL over the 2015-2020 period indicates that the Agency has effectively delivered the activities planned, often adding additional courses or increasing exchange opportunities to meet an increase in demand. As expected, the degree of implementation for residential activities and exchanges took a massive hit in 2020 due to the COVID-19 outbreak, which was only partially compensated by the increase of online activities. CEPOL’s training activities are considered of good quality by most of the stakeholders consulted through interviews and surveys and are deemed of poor quality only by a very small proportion (1%) of the surveyed training participants.

Table 12: CEPOL planned vs implemented activities

Sources: CEPOL CAARs (2015-2020), CEPOL Annual Report 2016.

Within the surveyed stakeholder groups, CEPOL’s residential training activities receive the strongest consensus regarding their quality, followed by seminars and conferences, and online training. The great majority of training participants found that CEPOL training meets their expectations and training needs under all aspects investigated by this evaluation (see Figure 2 below). The training offer is appreciated for its relevance and breath of topics covered, and for filling gaps and adding value to the training provided at the national level. The trainers and training materials are also considered of good quality by the surveyed training participants. Beyond the immediate learning benefits, CEPOL’s training opportunities are also widely appreciated for facilitating networking opportunities and strengthening cooperation with European counterparts and effectively building trust.

Several training participants satisfied with the quality of CEPOL’s training indicated that they would like to see CEPOL further increase the volume of activities and some would welcome a CEPOL certification system, as proof of the knowledge acquired. The latter appears as an objective in CEPOL’s programming documents, but has not yet been implemented. Practical training and experience sharing are typically used in CEPOL exchange programmes, which also received an overwhelmingly positive judgement from all surveyed stakeholder groups and interviewees. They stressed how operational collaboration is the most effective way for law enforcement officials to assimilate and internalise new practices.

The available information during the evaluation on the CEPOL European Joint Master Programme (EJMP) is too limited to provide a decisive judgement, however, the general perception from the few sources available is positive, except for one account, which lamented the absence of exams for assessing and certificating the knowledge acquired, which are, by comparison, available in the Frontex programme. The EJMP was discontinued in 2019 after its second edition due to issues in the selection procedures of the providers.

Figure 2: To what extent were you satisfied with the training opportunities offered by CEPOL between 2015 and 2020 in terms of:

CEPOL’s investment in upgrading and expanding its online training offer emerged as a positive strategy for extending its outreach and enabling a smooth transition to the delivery of training in pandemic times. However, the commitment to its own strategy was discontinuous over the years and by CEPOL’s own admission its online delivery tools fell behind those of other EU agencies. CEPOL tried to reinforce its digital offer through the new proprietary online training platform for Law Enforcement Education – LEEd, which went online in April 2020. All of CEPOL’s online activities are now supported by LEEd, which is open to all EU law enforcement officers and is free to use. The platform – which is now compatible with mobile devices – presents CEPOL’s training activities clustered according to thematic areas, course type, and topical tags. According to CEPOL’s Digitalisation Strategy 38 , LEEd will eventually address some of the requests of its target audience, including the categorisation of courses by level of difficulty, and a progression and certification system for the learner. In addition to the training area, LEEd provides a repository of articles from the International Police Science Journal (e-Journals) with browsing and filter capabilities by categories, a forum for the users to communicate, publish or modify content and a space for conference management. LEEd appears to satisfy the majority of training participants, and some explicitly say it represents an improvement from CEPOL’s previous platform ‘e-Net’ 39 , although opinions on LEEd are mixed. Some find it fit for purpose, user-friendly, and even eye-catching, while others say it is harder to use than its predecessor or find the module-based approach unintuitive and expressed dissatisfaction with content availability. Moreover, a non-negligible proportion of surveyed training participants indicated that they were unsatisfied with LEEd’s lack of simplicity and clarity of use, and the ineffectiveness of its search engine.

LEEd’s deployment encountered some challenges. It was delayed by four months and the migration of content from the previous platform suffered from further delays; some stakeholders also reported bugs and compatibility issues on some operating systems. The hosting infrastructure of the LEEd platform, managed by a German private company, suffered a security breach on 17 November 2020. While the overall balance of opinions depicts a positive user experience, LEEd’s cumulative issues have been an obstacle to the smooth delivery of online courses at a time when all of CEPOL activities moved online due to COVID-19.

Other issues identified related to the development and delivery of training activities outsourced to CEPOL’s FWPs, namely the short span of the yearly training development cycle and a blind spot in the quality assurance mechanism managed by CEPOL. As for the training development cycle, it was pointed out that once the thematic areas for training have been approved by the Management Board, the time at the disposal of FWPs to undergo grant procedures and develop course content is too short, effectively discouraging FWPs from applying and favouring those with off-the-shelf courses that happen to fit the chosen topics. As a matter of fact, the number of grant applications from FWPs does not meet the expectations. Outsourced training activities also suffer from uneven quality of delivery from FWPs, which CEPOL’s quality assurance mechanisms is currently unable to prevent.

Finally, the selection of the right participants for residential training is crucial to ensure that law enforcement officials gain the most out of CEPOL activities, in terms of exchange and relationship-building. However, stakeholders interviewed highlighted that participants selected for residential training do not always have a suitable profile in terms of the level of expertise and seniority. Activity Managers within CEPOL have the power to vet and decline participants’ applications, leaving the spot open for more suited candidates, but the evaluation finds that this power is hardly ever used, as it is an established practice to accept the first candidate presented by Member States, regardless of his/her profile.

The LETS Communication identified a number of training gaps that affected the effectiveness of cross-border cooperation across the EU: (i) a lack of qualified trainers and consistent learning materials on cross-border investigations; (ii) insufficient coverage in Member States’ training programmes of EU instruments and the EU dimension of day-to-day policing; and (iii) inadequate English language skills, considered a crucial competence for all law enforcement officials involved in cross-border cooperation. Evidence shows that CEPOL addresses all three gaps with dedicated training activities (see Table 1 3 ) and the development of ‘common curricula’, putting particular emphasis on training on EU instruments and EU policing.

Table 13: CEPOL activities to address training gaps identified by the LETS Communication

Source: CEPOL CAARs (2015-2019), CEPOL training catalogues (2015-2020).

The majority of respondents to the surveys were of the view that CEPOL effectively provides training to law enforcement trainers, and training participants have an overwhelmingly positive opinion of the trainers selected by CEPOL, suggesting that an increase of CEPOL’s offer in this respect would be positively received. Similarly, the majority of stakeholders believe that CEPOL has played a key role in shaping the tools and materials for law enforcement training, effectively filling gaps in training provided at the national level. Despite all of this, the majority of survey respondents within each group believed that training gaps between Member States have been reduced only ‘to some extent’. However, it should be noted that representatives from national police may not be able to offer a fully objective view of this issue, perhaps being inclined to overestimate the strengths of their national practices compared to alternative approaches, especially when it comes to national key areas of expertise. The analysis of available evidence shows that CEPOL is convincingly covering the right bases and is moving in the right direction to further reduce training gaps, but there is more distance to be covered to even out the field across the EU.

The 2015 CEPOL Regulation expanded CEPOL’s mandate with the responsibility to further coordinate training between EU agencies in the JHA area. Stakeholders interviewed recognised that CEPOL became more proactive in seeking to coordinate activities with agencies and actors in this policy sphere, organising annual meetings with both DG HOME and JHA agencies (including EASO, EIGE, EMCDDA, eu-LISA, Eurojust, Europol, FRA and Frontex) to synchronise training activities. However, a lack of sustained active collaboration within the JHA sphere (e.g. actors contributing to the EU JHA landscape that are not JHA agencies) was noted by some stakeholders, for instance with DG TAXUD, which coordinates training for customs and tax authorities in the EU. Furthermore, it was also suggested within the JHA area that CEPOL should reach out to other relevant groups that host law enforcement and prosecutorial networks, such as the Camden Asset Recovery Interagency Network (CARIN), although, progress was noted in forging cooperation with the European Network for Investigation and Prosecution of Genocide, Crimes against Humanity and War Crimes. Interviewees from JHA agencies unanimously considered CEPOL’s role to be complementary in achieving their objectives. Synergies and working relationships varied, centring on a number of key themes. Some JHA agencies provide training content to CEPOL, either by sharing their topic expertise to inform CEPOL modules in the absence of CEPOL in-house specialism, or by supplying experts to deliver training or speak at CEPOL organised webinars. Interviewees articulated that there is often a clear division of labour, with the JHA agencies responsible for the technical content of training and CEPOL providing administrative and organisational functions to disseminate and structure modules, including through the use of the LEEd platform. CEPOL provides a nexus and forum through which relevant JHA agency training can be disseminated to audiences beyond their traditional scope. Infrequently, CEPOL may take the lead in creating the content for a training course alongside JHA agencies. The effectiveness of CEPOL’s tools to pursue its coordination function is less certain. In undertaking this coordination function, CEPOL’s interactions with JHA agencies varies according to identified synergies and training needs of the relevant organisation. CEPOL previously made use of the JHA Training Matrix to identify training synergies. This methodology required other JHA agencies to list their specific activities, number of participants in attendance, and the target audience of them in a formatted Excel file, which had to be sent to CEPOL. When in use, the matrix was found to be an effective tool to map training activities, improve their coordinated delivery, help to reduce duplication of training activities.

Figure 3: In 2014 CEPOL introduced the JHA Training Matrix as a coordinated planning tool for training activities. To what extent do you agree with the following statements: (CNU/NCPs)

Source: Survey of CNUs/NCP (n=33).

In 2019, CEPOL decided to stop using the JHA Training Matrix, on the basis that it was not systematically completed by agencies, resulting in the collated information being incomplete. In an Audit on Needs Assessment conducted in 2016 it was reported that over half of the training activities recorded by the relevant agencies were uploaded after training had taken place, limiting the utility of the Training Matrix for planning purposes. The reasons for agencies not to complete the matrix may be linked to perceptions and experiences of the JHA Training Matrix being cumbersome to complete and ineffective. No structured methodology has replaced the JHA Training Matrix; instead, CEPOL undertakes coordination through a peer-to-peer approach. Some stakeholders considered this shift to ad hoc meetings as increasing the risk of future gaps and overlaps in training provision. According to stakeholders, initial requests from CEPOL for agencies to contribute content or speakers to training courses were sometimes received at short notice, limiting the Agency’s ability to fulfil such requests. As mentioned above, the working relationship and synergies between agencies varies. Whilst the majority of stakeholders agreed that CEPOL’s training activities were consistent and complementary with their own activities, the JHA Training Matrix may not have been the mechanism through which this coherence was achieved. Stakeholders reported that CEPOL is an active participant in the JHA Agencies Network. Ultimately, some stakeholders questioned whether CEPOL’s updated coordination role, and therefore identity, was made clear to all agencies and organisations, as they linked it to the perceived absence of strategic direction or guidance from EU legislative bodies in assisting CEPOL to fulfil this role, as CEPOL lacks authority to require agencies to cooperate. Nonetheless, this represents a concern that may undermine CEPOL’s external coherence in the long term. Some EU-level stakeholders and groups generally accorded with this view of CEPOL being an active cooperating partner when coordinating training at the EU level, 42 although the frequency and purpose of CEPOL’s communications varied across organisations. 43 The expansion of CEPOL’s activities to train, in collaboration with other entities, relevant cross-professional audiences – such as, for instance, prosecutors who authorise police use of investigative tools – was not perceived to have impacted upon training coherence in practice. Some stakeholder reported an increase in CEPOL’s courses produced with the Network of Public Prosecutors and the EJTN and the ENFSI, although the provision of topics relevant to judges is said to be limited. Stakeholders identified that prosecutors and judges may be more readily served by other bodies, such as the EJTN and training identification assisted by the European Training Platform.

The LETS Communication emphasised the importance of exchange programmes as a key instrument to increase cooperation, share good practice and build trust among EU law enforcement officials. CEPOL responded with a progressive increase of exchange opportunities over the evaluation period both by intensifying exchanges, and by extending exchange opportunities to other stakeholder categories and partner third countries within the scope of CEPOL’s projects in the EU neighbourhood. Stakeholders’ opinion on the CEP is overwhelmingly positive. The exchange programmes emerged as clear favourites among the interviewees as well, which stressed how operational police work conducted during exchanges is the most effective way to build a trusting relationship between law enforcement services.

Exchange programmes are not the only activity conductive to cooperation and trust-building. The majority of all consulted stakeholders agree that CEPOL activities as a whole effectively led to an increase in mutual trust among law enforcement organisations. Moreover, stakeholders believed that CEPOL’s activities facilitated an increase in the number of opportunities for cooperation and in the quality of cooperation between national law enforcement services at EU level as well as, to a smaller extent, on a bilateral and regional basis. The majority of training participants agree that CEPOL’s training activities provide concrete opportunities to establish collaborations with their European counterparts and that CEPOL adds unique value to develop cooperation that would not have otherwise been achieved through national means. One interviewee underlined that, because of the nature of their work, law enforcement officials are naturally reluctant to trust and share information with people they do not know, but meeting their counterparts from other Member States during CEPOL’s activities contributes to reduce the barrier of mistrust. Several stakeholders also stressed that moving training activities online due to COVID-19 (which was a positive and responsive step by CEPOL given the circumstances) meant losing one of the most impactful aspects of CEPOL’s activity: networking and relationship building, deemed crucial for effective cooperation. Stakeholders with direct knowledge of CEPOL’s external activities consider that CEPOL’s activities are an important tool for improving relationships between the EU and third countries, in particular, to harmonise investigation and prosecution practices with EU standards and to reinforce cooperation among partner countries with positive effects on regional stability and enhanced dialogue between law enforcement institutions in the EU neighbourhood.

The evaluation has considered to what extent the current governance structure of CEPOL has brought positive effects to the overall functioning of the agency. Case study 1 44 addresses aspects of CEPOL’s governance and internal workings in more detail. Evidence suggests that the new governance structure had positive effects on the Agency’s ability to fulfil its activities and objectives. The 2015 evaluation of CEPOL noted that the relationship between the Management Board (formerly known as the Governing Board) and the Agency was disproportionately skewed towards the former, as Board Members were entirely responsible for developing CEPOL’s strategic direction and the Agency only had an administrative and supporting role. The current governance, which reinforced the role of the Executive Director, strikes a good balance of responsibilities between the two, in the opinions of the overwhelming majority of stakeholders. Thus the Executive Office is fully dedicated to the Agency and can partly compensate for some of the issues affecting the Board identified in the course of the evaluation, namely the turnover within the Board and the low level of engagement of a non-negligible part of Member State representatives. It emerged that the Board consistently suffers from low levels of participation from several Member States, an issue already present in the pre-2015 era and which according to these stakeholders remained unchanged by the introduction of the 2015 Regulation. The Executive Director emerges prominently as the driving force within the Board, on one hand compensating for less engaged Board Members, but also further disincentivising participation. There are also issues around representation on the Board, with Member States sometimes appointing persons that are unqualified or unable to commit to the job, and the composition of the Board does not currently represent CEPOL’s target audience; furthermore, the Board also consistently lacks the balance in gender representation required by Article 8 of Regulation 2015/2219, as only 35% of NCPs are women (9 women out of 26), and the unbalance is only slightly reduced to 38% of women when accounting for alternate voting members (19 women out of 50). The agenda of Board Meetings being cramped with administrative items is also recognised by some interviewed as an issue affecting the room for strategic discussion, although CEPOL has already tried to tackle this issue by regularly using the written procedure for administrative decisions and by encouraging an increased participation of Board Members, also through Strategic Discussion Groups. Thus the great majority of stakeolders consider CEPOL decision-making procedures to be effective, despite the issues identified.

Communication between the different components of CEPOL’s governance appears effective. The introduction of CNUs improved CEPOL’s ability to collect the necessary information from dedicated operational interlocutors at the national level. However, some NCPs feel they are overloaded with information, which they are able to digest only through the filter of CNUs and pointed out a bottleneck in communication at the national level. This is due to the fact that CNUs are organised differently in each Member State (as permitted by the 2015 Regulation), and in some national setups the NCP and the CNU have been selected from different institutions with hardly any contact, affecting the awareness and preparation of some NCP during Board Meetings. High staff turnover among CNUs and CEPOL has led to the loss of consolidated relationships between CEPOL staff and Member States representatives and a general difficulty in forming new ones, leading to stiffer and somewhat less efficient communication.

The analysis has also found that most of the provisions of Regulation 2015/2219 are aligned to the requirements of the Common Approach 45 , and it appears clear that the Common Approach informed the revision of CEPOL’s Regulation in 2015, as illustrated in case study 1 46 , with the notable exception of the Executive Board, which was left out of the 2015/2219 Regulation. An analysis of the compliance of the main provisions of CEPOL’s legal basis defining the Agency’s internal structures, governance, administrative and financial rules is presented in Table 1 4 below.

Table 14: Matrix of correspondence between CEPOL's regulation and working practices and the requirements of the Common Approach

Notes: Areas shaded in green indicate compliance, while areas marked in amber indicate non-compliance.
Source: Research team’s construction from available evidence.

CEPOL has a quality assurance mechanism in place to control its training activities from development to delivery. The evaluation found that CEPOL’s quality control is effective in guaranteeing high-quality content and trainers when CEPOL is in full control of this process. However, training activities outsourced to FWPs, while still appreciated by participants, suffer from instances of variable quality that escape CEPOL’s quality assurance mechanism. CEPOL recognises the issue and tried to address it through a new centralised business model which has not found consensus within the governance. CEPOL opted then to adopt the current “hybrid” model which relies on both outsourcing training activities to FWPs, and centralising training development through the CKCs. In the 2015-2020 period, CEPOL invested part of its resources in assessing the quality of its training activities and the performance of trainers, as required by the LETS and recommended in the 2015 evaluation of CEPOL. Documentation on evaluated activities provided by CEPOL confirms that all of its activities – either delivered within the EU or in third countries – do, at a minimum, undergo an evaluation of participants’ satisfaction based on Kirkpatrick’s model for training evaluation. However, the Kirkpatrick’s model appears to be more of a tool for categorising the levels of training knowledge rather than a practicable method to evaluate them. In order to robustly measure the effectiveness of CEPOL’s training offer in improving the capabilities of law enforcement in the EU (in line with the objectives of the Agency), methodologies should ideally be experimental (e.g. pre-post design comparing outcomes among a group of law enforcement officials that are trained by CEPOL and those who are not). However, the research team acknowledge that measuring the impact of training beyond satisfaction and knowledge retention is resource-dependent and methodologically challenging, as also pointed out in interviews.

As for internal control systems, CEPOL has been using a custom version of the European Commission Internal Control Framework (ICF), adapted to the CEPOL context and which complies with 13 ICF standards and CEPOL is certified in accordance with the Quality Management System Standard ISO 9001:2015 since 2017.

The only external factor influencing CEPOL’s effectiveness that emerged prominently from the analysis is the multiplicity of training providers that compete for the attention of the law enforcement target audience. At the EU level, as discussed in case study 3 48 below, CEPOL shares segments of its audience with several other EU agencies that run their own training units and have developed separate training platforms. While there is evident room for improvement, the EU space is one that incentivises coordination, and all JHA agencies interviewed considered there to be good synergies between their own training offering and that of CEPOL. Still, the time law enforcement officials can dedicate for professional development activities is a finite resource, and CEPOL’s outreach numbers are inevitably influenced by the availability of alternative options provided by other JHA agencies. In the EU neighbourhood, as discussed in case study 2 49  below, the level of competition is much more pronounced. Not only does CEPOL have to compete for its target audience with international organisations and providers from other third countries with political interests in the partner countries, but with the bilateral initiatives of Member States as well. While all consulted stakeholders believe that coordination among different providers has much improved over the years, turning some of the duplication into synergies, the ‘training market’ in third countries appears significantly oversaturated and there is room for improvement, at least between CEPOL’s activities and the Member States’ initiatives.

Some issues and areas for improvement emerged from the analysis:

o    The results of CEPOL in certain areas suffer from under-exposure or low recognition within CEPOL’s own governance (i.e. support to bilateral/regional cooperation, and support to capacity-building in third countries).

o    The Operational Training Needs Assessment (OTNA), which builds upon the EU-STNA, is suffering from a low response rate, resulting in less relevant operationalisation of selected training topics.

o    CEPOL’s training methodologies could be further tailored to its target audience as law enforcement officials respond better to hands-on practical training (joint exercises, simulations, table-top exercises) rather than (virtual) classroom theoretical training.

o    The newly implemented e-learning platform LEEd was delayed, albeit the user experience appears generally positive.

o    The possibility to manage Union External Assistance funds introduced in Regulation 2015/2219 proves crucial as CEPOL must rely on external funding from Commission programmes to fulfil its objectives in support of the EU external action.

oAlthough CEPOL has been effective at addressing the gaps in law enforcement training, there is room for bringing the Member States’ training systems closer together, as stakeholders still recognise disparities in the level of maturity of national training systems.

oNo structured methodology has replaced the JHA Training Matrix; instead, CEPOL undertakes coordination through a peer-to-peer approach.

oAccounts of cooperation resulting from CEPOL’s activities are anecdotal since national law enforcement services do not systematically record or measure their occurrence or impact. CEPOL does not measure these outcomes either: while CEPOL’s mandate prescribes the agency to evaluate its training, the agency has focused on measuring participants’ satisfaction and learning outcomes.

oThe Management Board suffers representation issues in terms of engagement across Member States, gender balance, and CEPOL’s enlarged target audience.

•    CEPOL struggles to attract and retain suitably qualified personnel as it is not a strong competitor on the jobs market; the Agency considers that this is mainly due to the low salary correction coefficient.

•    CEPOL’s current business model to outsource part of the training activities to Framework Partners in the Member States weakens the quality assurance mechanism.

•    CEPOL competes with several training providers (other EU agencies, international organisations, foreign governments, Member States’ bilateral initiatives) for the attention of its target audience both within the EU and in third countries.

5.2. Efficiency

This efficiency criterion demands an assessment of the relationship between the cost of operating CEPOL and the changes it has led to (or contributed to) because of its activities. For this evaluation, the benefits of CEPOL’s activities cannot be quantified so the focus is on the costs of CEPOL’s activities and the efficiency of CEPOL’s operations. Efficiency is a measure of how resources/inputs (funds, human resources, expertise, time etc.) are converted into results.

This evaluation has considered the costs incurred by CEPOL and its broader stakeholders in the delivery of its activities, as well as any potential benefits generated. For the purpose of the evaluation, the costs have been aggregated at a high-level (broken down by operations, staffing and building and equipment to be consistent with CEPOL documentation and reporting, rather than by activities). The inability to quantify the benefits means that this analysis is unable to make any concrete conclusions about the cost effectiveness of the Agency.

As shown in Table 3 above, CEPOL’s over the 2015 to 2020 period, operating costs comprised on average 54% of the budget, staffing costs comprised 41% and other costs comprised 5% 50 . The overall number of CEPOL staff has increased steadily over the period 2015 to 2020 from 45 to 84 (see Table 2 ). CEPOL funds Member States’ representatives to participate in CEPOL activities through annual allocation of grants to Framework Partners selected via restricted calls for proposals every four years. These grants typically cover travel, accommodation and catering for Member State representatives to attend CEPOL activities. On average it sems that €5,000 in additional costs are incurred for hosting each CEPOL activity. This variation between Member States can be explained by the fact that Member States’ available resources and infrastructure to host CEPOL activities can vary. Member States fund the cost of staff within the CNU who act as a liaison point for law enforcement officials wanting to participate in CEPOL’s activities. There may be other ‘soft’ costs associated with Member States’ participation in CEPOL activities, but it is difficult to quantify these costs and they likely vary to a great degree across the EU. Most of the CNU/NCPs, FWPs representatives, training participants and third country representatives that responded to the surveys agreed that the benefits stemming from participating in CEPOL’s activities outweigh the costs for Member. These benefits of CEPOL are difficult to quantify, though the evaluation found that the overall outputs of the Agency are valued by stakeholders at the EU/national/international levels. The evaluation considers whether CEPOL could have achieved greater benefits with the same level of human and financial resources, and whether CEPOL’s governance structure, organisational set-up, management systems and working methods have contributed to the Agency’s efficiency, or whether there have been any delays or bottlenecks in these internal structures that may have negatively influenced efficiency. The stakeholders generally perceived that CEPOL’s resources were managed well and used in the most efficient way possible for achieving the Agency’ objectives, with only a minority of respondents indicating a partial efficiency in the use of resources. As one measure of efficiency, the research team calculated a high-level estimate of the cost per training participant over the period 2015 to 2020, to interpreted with caution as they rely on crude assumptions. There appears to be a downward trend in the cost per participant, which is consistent with the trend observed in the previous evaluation commissioned by CEPOL and the views of stakeholders.

Table 15: Estimated cost per training participant

Source: Operating costs were from CEPOLs CAARs (2015 – 2020) (as presented in Table 3 ); and participant information is taken from CEPOL CAARs (2015, 2016, 2017, 2018, 2019, 2020). Notes: Average cost per participant is the research team’s calculation, where C = A / B. When calculating estimated cost per training participant from CEPOL’s total costs (operating, staff and administrative) the trend is consistent (i.e. 2015: 1,912, 2016: 487, 2017: 394, 2018: 359, 2019: 268, 2020: 264).

Representatives of the Management Board, complimented CEPOL’s internal functions for its efficient communication between staff and strong leadership from the Executive Director, while an EU-level representative reflected on the overall professionalism of CEPOL. 51 Nevertheless, results from the 2019 CEPOL Staff Engagement survey highlight that some CEPOL staff shared less positive views about the Agency’s internal communication (see case study 1 52 ). Despite clear internal structures, there was more disagreement observed amongst survey respondents in relation to the efficiency of the CEPOL Management Board decision-making, as well as CEPOL’s planning processes and monitoring procedures. Notably, two stakeholders highlighted that the Management Board’s deliberations and decision-making process was often slowed down by some representatives’ lack of awareness of the EU law enforcement landscape and variance in terms of Member States’ participation in the meetings. According to CEPOL staff, CEPOL HQ’s internal structure appears to operate efficiently. The Agency’s move towards greater digitalisation of procedures (starting prior to the 2015 Regulation) was seen as a positive development. Nevertheless, CEPOL’s staff turnover might have affected the Agency’s overall efficiency during the 2015 to 2020 period. CEPOL has experienced a high staff turnover in the past five years, which the Agency attributes to the relocation from the UK to Hungary in 2015 and the low country coefficient applied to staff salaries. According to CEPOL’s CAARs, this relocation has not assisted CEPOL in attracting staff from the broadest possible geographical basis among the Member States (see also case study 1 53 ). In 2020, Hungarian citizens represented 40% of all staff employed at the Agency. As of 31 December 2020, CEPOL does not employ staff from Bulgaria, Cyprus, Czechia, Denmark, France, Ireland, Luxembourg, Malta and Slovenia, which reflects the lack of applications from suitable candidates in these countries. This point was raised also by the European Parliament Committee on Budgetary Control. Despite some mitigating actions being implemented, the Committee reported concerns on the possible impact on business continuity and the Agency’s ability to implement activities. Moreover, the disproportionate number of Hungarian staff should be interpreted in the context of the EU’s emphasis on attracting a diverse workforce and fostering diversion in selection and recruitment procedures. These findings are consistent with concerns raised as part of the previous evaluation of CEPOL (see case study 1 54 ). One CEPOL representative stressed that CEPOL’s internal decision-making process was sometimes hampered by the Agency’s complex management structure, which required multiple parties to approve decisions. The administrative burden on Member States was highlighted as an issue by stakeholders in the previous evaluation. CEPOL now operates under a hybrid business model, which came about further to the Member States’ rejection of a totally new business model, meant to streamline and simplify the grants management. While the majority of stakeholders agreed that the hybrid model improved the quality of training activities and helped produce a comprehensive multiannual portfolio, when asked questions relating to internal efficiency, the opinion was less positive. A relatively small share of respondents strongly agreed that the model’s capacity reduced the administrative burden on Member States and increased the efficiency of CEPOL in organising and assigning training. Overall, survey data showed that CEPOL’s Framework Partner representatives were satisfied about their working relationship with CEPOL. The majority of respondents strongly agreed that i) the selection process for Framework Partners was clear and streamlined; ii) the roles and responsibilities of Framework Partners is clearly defined and allocated; and iii) CEPOL’s procedures and decision-making process for awarding grants to Framework Partners is sufficiently fast and transparent. Yet, most Framework Partner respondents stated that CEPOL’s monitoring and reporting requirements could be streamlined and simplified. The survey data also highlighted that some of the roles and responsibilities of different national actors interacting with CEPOL (including Framework Partners, National Units and NCPs) might be somewhat unclear. The evaluation considers whether political, economic, social and cultural, or technical factors at the EU or Member State level may have affected the efficiency of CEPOL. The only external factors mentioned by stakeholders as having the potential to impact on CEPOL’s efficiency in the last five years and also representing an ongoing threat was the COVID-19 pandemic. This evaluation has considered whether the resources allocated to CEPOL have been adequate for the Agency to carry out its mandate since 2015. There was no significant increase in CEPOL’s EU annual subsidy in the period before and after the new legal basis came into effect in 2016, despite the fact that CEPOL’s mandate was expanded. The Internal Audit Service (IAS) Audit Report from 2020 highlighted that an inherent risk for CEPOL is the “lack of agreement between the Agency and the European Commission regarding the resources necessary to fulfil its mandate.” Consistent with this, a number of Member State-level, JHA agency and EU-level representatives raised these concerns, similarly to the previous evaluation of CEPOL. Many of the respondents did not consider CEPOL’s current resources to be sufficient in light of emerging and new challenges facing the Agency. The evaluation identified two areas where there was compelling evidence from both desk and field research that additional resource could have further enhanced CEPOL’s ability to meet its objectives during the period from 2015 to 2020: CEPOL’s ability to reach a larger target audience and CEPOL’s information and communication technology (ICT). The analysis of the sufficiency of CEPOL’s resources also takes into account CEPOL’s budget implementation. As shown in Table 16, it can be seen that CEPOL was capable of absorbing the funding provided to the agency by the EU.

·

Table 16: Budget implementation

Source: CEPOL’s CAARs 2015 – 2020, CEPOL Report on budgetary and financial management for 2020 (art. 9), the evaluation team notes that the summarised data was kindly provided by a representative from DG HOME.

5.3. Relevance

The relevance criterion examines the extent to which CEPOL’s activities are in line with the needs of the EU, Member States and its external stakeholders. It examines the relationship between CEPOL’s work programmes and the needs and problems that existed in relation to the need for coordination in law enforcement training. The evaluation of the relevance of CEPOL has taken into account the Agency’s objectives set out in Article 3 of Regulation 2015/2219 vis-à-vis EU needs and challenges as defined by the Stockholm Programme and the priorities listed in the EU Agenda for Security 2015-2020 (hereinafter the Agenda), as well as the European LETS. Overall, the evidence collected suggest that CEPOL’s objectives are relevant to EU needs and challenges.

CEPOL’s first objective – to support Member States in the provision of training to law enforcement officials, in particular on elements of cross-border and European-level policing – appears to be aligned with the EU needs and challenges stemming from the Stockholm Programme and Agenda, and addressed through a series of activities implemented by CEPOL, such as:

·Training on the implementation and use of international and Union instruments for law enforcement cooperation, which directly address the need to foster cross-border cooperation between Member States’ law enforcement services.

·Training on Union bodies and JHA agencies, which responds to the need expressed in the Agenda to improve inter-agency and cross-sectoral approaches to maximise their contribution to Member States and EU action.

·Training on judicial cooperation and practical use of information exchange channels, which directly addresses the need to ensure that Member States    fully exploit existing tools for information sharing.

Stakeholders did not identify a comparable training provider at the EU or Member State level that offered training on EU-wide issues and EU-wide instruments for law enforcement cooperation.

The second objective – to support Member States in the development of bilateral and regional cooperation – was not present in CEPOL’s legal mandate prior to the 2015 Regulation. The evaluation found that CEPOL’s objective is relevant to EU needs in so far as it contributes to achieving greater cooperation among EU law enforcement services and the exploitation of available instruments such as the European Arrest Warrant, 55 rogatory letters 56 and readmission procedures. However, none of the stakeholders explicitly identified the development of bilateral and regional cooperation among Member States as one of CEPOL’s distinct activities, and thus did not comment on its relevance in those terms, but it was mainly recognised, as a by-product of the networking and trust-building that results from CEPOL’s residential training and exchange programme.

CEPOL’s third objective is to develop, implement and coordinate training addressing specific criminal or policing thematic areas. The Agenda indicates that CEPOL should adapt its yearly training programmes to the priorities set at the EU level. Since 2010, serious and organised crime priorities have been defined at the EU level through the EMPACT, which aims to reinforce European law enforcement’s expertise and cooperation to adequately respond to identified criminal threats. CEPOL’s objective to develop, implement and support training on specific thematic areas appears relevant to EU needs as it enables the Agency to stimulate and coordinate the production and delivery of training on the EU’s top priorities. A review of CEPOL AWPs showed that CEPOL gives priority to training activities stemming from the priority thematic areas of EMPACT and the Agenda, highlighting CEPOL’s commitment to addressing EU needs and challenges. Stakeholders reaffirmed how the ability of CEPOL to adapt to new trends and respond to the requests for training on specific thematic areas is crucial to remain relevant as a training provider.

CEPOL’s expanded mandate to support the training of law enforcement officials when preparing for deployment in EU missions (i.e. CSDP) appears to be in line with the need expressed in the Agenda to maximise JHA agencies’ contribution to EU action and security objectives 57 . CEPOL shares the mandate to deliver pre-deployment training for CSDP missions with another EU-level training provider, namely the European Security and Defence College (ESDC). However, case study 2 58 found that the two could cover different aspects of CSDP related training, namely the ESDC is better suited to cover pre-deployment training, while CEPOL has an edge in delivering training on specific crime topics relevant for civilian CSDP missions. There is no uniformity of opinions about the overall relevance of this objective. For some stakeholders, CEPOL’s involvement is expected and necessary; for others, ESDC’s activities are sufficient to cover pre-deployment training and CEPOL could devote its resources to providing training on topics for which there is a higher demand.

CEPOL’s objective of supporting the training of law enforcement officials in capacity-building activities in non-EU countries appears relevant to the needs and challenges of the EU, also in line with the 2009 Stockholm Programme, the European Agenda on Security. The stakeholders highlighted the important role of CEPOL in this area and confirmed its relevance. They underlined how expanding the mandate was an important step and that CEPOL is positively perceived in third countries, becoming the “passport” for the EU when trying to unify the law enforcement culture and reinforce cooperation beyond its borders. A few interviewees, however, expressed the opinion that CEPOL’s involvement and presence is still too limited (e.g. compared to Europol in areas such as the Western Balkans) and how these activities may draw attention (and human and financial resources) from other activities related to the “core business” of the Agency.

Concerning the objective to provide training to law enforcement trainers and support the development and exchange of best practice regarding law enforcement practices and techniques, the analysis shows how it is clearly aligned with the EU needs expressed by key strategic documents such as the LETS Communication. The development and exchange of best practice has been central to CEPOL’s activities since it was established, together with the development of dedicated training for trainers – however, the number of these activities is still quite low, especially if compared to other JHA agencies. The evaluation confirmed that CEPOL is uniquely positioned to collect and disseminate knowledge on law enforcement practices among Member States and that CEPOL is well positioned to embed best practices into training by training European law enforcement trainers. Some stakeholders also share the view that the quality of CEPOL’s training is heavily contingent upon the quality of trainers, suggesting that improving trainers’ skills is highly relevant.

The objective to develop, upgrade and evaluate learning tools and methodologies that support the learning and development of law enforcement officials regarding law enforcement practices and techniques was introduced by Regulation 2015/2219, raising the quality requirements expected from CEPOL, as recommended in the LETS. The document also called for the “need to ensure high quality of delivery” of law enforcement training by defining EU training strands and by stepping up the development, delivery and evaluation of the training and learning opportunities needed to meet the EU’s cross-border law enforcement challenges. The 2015 CEPOL evaluation recommended that CEPOL invests in evaluation tools, especially to vet trainers and measure the effect of the training courses on participants’ level of knowledge, instead of limiting the evaluation to participants’ satisfaction levels. However, some interviewees who mentioned this topic during the conversation did not refer directly to CEPOL as the body responsible for implementing monitoring and evaluation practices, but highlighted how relevant evaluation methodologies were developed by their own organisation. Moreover, the fact that CEPOL still relies heavily on satisfaction rates to assess the quality and impact of training does raise some limitations on this point, also with the Kirkpatrick model being more a categorisation tool rather than a realistic evaluation method.

The evaluation found that CEPOL’s objectives are relevant to stakeholders’ needs; the training offer has generally responded to the training needs of the Agency’s target audience over the 2015 to 2020 period. While there are differences between the views expressed by different categories of stakeholders responding to the survey, and between objectives for the same stakeholder, there is no case where the majority of the stakeholders involved deemed an objective to be “not relevant” for their needs.

Figure 4: To what extent were the following objectives of CEPOL (set out in Regulation (EU) 2015/2219) relevant to the needs of your country between 2015 and 2020?

Source: Survey of CNUs/NCP (n=33)

Some stakeholders raised doubts over the full relevance of the objective to support the training of law enforcement officials when preparing for deployment in EU missions, as they were of the view that CEPOL’s resources should be devoted to providing training on topics where it is more needed. The introduction of EU-STNA and OTNA was an important step forward in ensuring a stronger alignment of training topics and activities to stakeholders’ needs, according to stakeholders. Participants to CEPOL’s training activities responding to the survey underlined how CEPOL managed to integrate its training offer with courses on topics that responded to specific needs and priorities. However, some stakeholders reported the limited number of respondents to the OTNA and that the training offer somewhat omitted certain areas or target groups (i.e. forensics, customs, judges). Furthermore, for the majority of Framework Partners 59 answering the survey, the instrument improved prioritisation only to some extent, suggesting the existence of room for further improvement.

Results of and feedback on the functioning of EU-STNA may need to be reviewed in the light of the implementation of the next, revised cycle 2022-2025.

According to Regulation 2015/2219, CEPOL's activities should: (i) focus on priorities and areas where training can add value for Member States and the Union; (ii) address Union priorities in the area of internal security and its external aspects, in line with the relevant policy cycles; and (iii) ensure that training needs assessments are part of its planning to enhance the effectiveness of future actions 60 . This evaluation finds that CEPOL makes tangible efforts to meet these objectives and maintain relevance in doing so. There were mixed findings in relation to CEPOL’s ability to adapt its training offer to scientific, technological and socio-political developments. The content of CEPOL’s training catalogues is depicted in Table 1 7 below.

Table 17: Key developments considered in CEPOL’s training offer

Source: CEPOL’s training catalogues (2015 – 2019).

Several stakeholders mentioned how CEPOL has been able to adapt its training activities to new developments, priorities and challenges faced by law enforcement, including under COVID-19. These efforts resulted in the production of 25 COVID-19 related webinars (as of 28 February 2020), addressing a mix of new issues that emerged during the pandemic, and COVID-19 driven developments in relation to existing issues. On the other hand, respondents to all four surveys consistently indicated that CEPOL managed to adapt only “to a certain extent” to relevant developments. Only in a few cases, however, were they able to suggest further developments (see Table 1 8 ), but also from Table 1 7  it emerges how the coverage of some technological developments is not continuous (e.g. drone technology, 3D printing, AI, IoT), or not present at all (e.g. 5G).

Table 18: List of topics and developments highlighted by stakeholders as not duly taken into account by CEPOL’s training offer 61

Source: Survey of CNUs/NCPs (n=33) and of participants (n=769).

Concerning political priorities, aligning CEPOL’s activities to the priorities for the EMPACT and the use of EU-STNA and OTNA are two elements that indicate a positive response from the Agency in this sense. Concerning technological developments, CEPOL has upgraded its training delivery platforms, with the transition from e-NET to the LEEd platform, launched in April 2020, although its implementation experienced some challenges and CEPOL representatives themselves indicated that whilst LEEd was a step in the right direction in terms of digital delivery, it is still not enough and a clear strategy is needed to understand the right mix of residential and online training for the future. Finally, particular reference was made by several stakeholders to CEPOL’s efforts to integrate cybercrime-related issues in its training offer. Most training participants responding to the survey agree that training activities in this area were adequate, even if the need for training remains higher than what CEPOL offers/can offer. Concerning the Cybercrime Academy established by CEPOL, stakeholders generally welcomed its creation as a good; however, a few interviewees expressed more critical opinions on the actual implementation of the Academy. It was pointed out that it cannot be considered a real “academy” as it does not produce research, and the number of staff working in the Academy may be rather low, raising doubts also about the ability of CEPOL to attract the best experts in the field. It remains, according to these stakeholders, a place where cybercrime can be discussed, but not taught or trained. According to stakeholders interviewed, CEPOL could potentially build expertise in trainers through train-the-trainers programmes and/or by drawing from the private sector. Moreover, the Academy cannot be accessed during COVID-19 pandemic, and, the fact that is it hosted in different premises other than CEPOL headquarters, posed some challenges especially from IT and document workflow perspective.

Some issues and areas for improvement emerged from the analysis:

o    Stakeholders raised questions about the full relevance of certain objectives, such as support the training of law enforcement officials when preparing for deployment in EU CSDP missions for CEPOL.

o    CEPOL strongly relies on qualitative assessment such as satisfaction rates to assess the quality and impacts of training.

o    According to some, the objectives of supporting Member States in developing bilateral and regional cooperation through the delivery of multi-lateral training and supporting the training of law enforcement officials in capacity-building activities in non-EU countries have a relatively lower relevance for CEPOL compared to other objectives.

o    Not all developments have been fully considered in the training offer.

o    Despite the positive assessment of the EU-STNA and OTNA process, the low participation rate in the OTNA is a concern.

o    The Cybercrime Academy received some mixed reports from stakeholders (no research conducted, limited size and ability to attract experts/skills).

5.4 Coherence

The coherence criterion assesses the extent to which activities and objectives of CEPOL are: (i) consistent, logical and not overlapping internally, and (ii) seek to achieve common objectives with those of external stakeholders (i.e. Member States, the Commission, EU agencies and non-EU agencies).

The majority of stakeholders surveyed or interviewed considered the roles and responsibilities of staff within CEPOL to be internally clear, defined, and thus coherent. However, whilst different agencies reported CEPOL to be responsive and communicative, this did not always result in the effective resolution of ongoing operational issues, for example, around the compatibility of CEPOL and other agencies’ online training platforms. Around two-thirds of CNU/NCP survey respondents reported that CEPOL’s planning and monitoring processes could be streamlined. Given CEPOL’s comparatively small size, there is arguably reduced opportunity for overlap in its internal functions. However, evidence highlights that there may be a disconnect between communications and directions from the Management Board and the staff responsible for putting procedures into practice, in the absence of further middle-management level staffing. One stakeholder reported that the number of staff at CEPOL is skewed towards the Executive Director’s office or the Head of Operations function, resulting in the absence of direction through line management for colleagues. This was said to result in staff having to figure out the remit of their role and act accordingly, with a lack of a clear strategy from the Management Board as to how CEPOL will evolve its law enforcement training offering. This was also evidenced in CEPOL AWP documents, with the grading of staff considered to be ‘low’ as compared to their responsibilities. This was reflected in the 2019 CEPOL Staff Engagement Survey (SES), which identified room for improvement on aspects such as leadership and staff motivation and morale. The Internal Control Framework (ICF) is seeking to enhance CEPOL’s internal coherence, meant to support internal decision-making and adherence to relevant rules and regulations. This new framework is said to reflect the Commission’s ICF Framework and gives managers ‘necessary flexibility’ whilst ensuring ‘robust internal control’ in processes and activities that are consistent throughout the organisation. Progress against the ICF is monitored through a mix of targets and ‘soft’ controls, such as morale, integrity and leadership. CEPOL’s 2019 CAAR identified internal control deficiencies relating to a lack of formalising controls over technology (i.e. ICT Backup Policy and disaster recovery plan from ICT perspective, Information System Security Policy), and procedural framework for document management. One stakeholder suggested that CEPOL does not have the resources to fill these positions. This suggests that, despite an updated coherent framework to instil processes, there might be resource gaps in being able to implement this. Interview and survey data indicate that CEPOL’s remit and activities are perceived to be coherent with and complementary to other relevant actors at the EU level. Evidence from the evaluation shows that CEPOL’s relationship with each relevant agency and actor is distinctive, signalling that CEPOL may adapt its working approach to maximise potential synergies. That said, CEPOL’s coordination functions are more reactive than proactive, with the absence of a structured methodology to map training content across the EU landscape in a coherent manner, as mentioned above. The evaluation has considered the extent to which CEPOL’s objectives and activities are consistent and aligned with those of other EU level training providers (e.g. the EJTN) and those delivered by relevant agencies, e.g. within the JHA Agencies Network. The evaluation has also tried to assess the alignment of CEPOL’s activities with projects and entities developing specific training materials (e.g. ECTEG). Overall, there is good coherence observed between the 2015 Regulation and the requirements to deliver training across other organisations. Whilst the expansion of the CEPOL target audience in the 2015 Regulation may introduce perceived overlaps, JHA agency stakeholders reported that each agency’s core focus and insights ensure that training content is distinctive and tailored according to their respective audiences. Survey findings suggest that CEPOL’s training activities are generally perceived to be coherent and complementary with training offered by national training providers. Some Member State and EU-level stakeholders were critical of CEPOL’s cybercrime training offering, questioning the quality, consistency and breadth of the training provided.

Figure 5: What are your views on CEPOL’s Cybercrime Academy and its role as providers of cybercrime training activities? (CNUs/NCPs)

Source: Survey of CNUs/NCP (n=33)

Figure 6: What are your views on CEPOL’s Cybercrime Academy and its role as providers of cybercrime training activities? (Training participants survey)

Source: Survey of participants (n=769)

Interpreting these results, it could be that rather than potential duplication, there may be gaps in CEPOL’s cybercrime training (in terms of identifying relevant stakeholders and existing content/materials) as compared to training developed within the Member States (specifically within the private sector). One stakeholder noted that despite there being good knowledge on cybercrime (applicable to training) within the Member States, CEPOL training represents the EU-level perspective, which is valuable in supplementing Member State level knowledge.

Some issues and areas for improvement emerged from the analysis:

·There may be a disconnect between communications and directions from the Management Board and staff responsible for putting procedures into practice, in the absence of further middle-management level staffing to oversee this.

5.5 EU added value

The EU added value criterion aims to identify the extent to which CEPOL has brought benefits in comparison with what exists at EU and national levels. The criterion assesses the extent to which CEPOL has helped to improve the coordination of police training.

This evaluation has considered whether CEPOL’s objectives and tasks have led to an enhanced ability of Member States to train their law enforcement officers, and the extent to which any benefits from CEPOL’s activities are at least in part attributable to their EU-level character.

The evaluation found that CEPOL’s objectives and tasks have offered added value compared to the law enforcement training offered by the Member States. The majority of stakeholders that responded to the surveys (including CNU/NCPs, FWPs and training participants) and public consultation stated that CEPOL added at least some value across CEPOL’s key objectives. Whilst the numbers were not large, it is worth noting that some CNU/NCPs responding to the survey claimed that CEPOL ‘added no value’ in the following areas:

·Supporting bilateral and regional cooperation (5 respondents);

·Supporting capacity building in third countries (3 respondents);

·Providing training to law enforcement trainers (2 respondents);

·Developing, implementing and supporting training on specific thematic areas (2 respondents);

·Preparing officials for EU mission (2 respondents).

As mentioned in case study 2 62 , the evaluation found that some stakeholders lack awareness about CEPOL’s capacity building activities carried out in third countries, which might explain why they are perceived as adding less value compared to other objectives. In fact, many respondents across the surveys and public consultation responded ‘I don’t know’ when asked about CEPOL’s added value in providing capacity building activities in third countries. The research team did not have access to Member States’ training offer to analyse whether CEPOL’s offer added value compared to that offered by Member States. However, stakeholders interviewed in the context of this evaluation agreed that CEPOL’s objectives and tasks helped to improve the ability of Member States to train their law enforcement officials compared to what they could have done at the national level. Some interviewees stated that CEPOL usefully offered training on EU-level instruments and tools and provided insights into the EU dimension of cross-border issues, which were seldom covered in national training. This is all the more a valid point for smaller Member States that do not have the resources to provide tailored training on EU-level JHA issues. According to the interviewees, CEPOL’s greatest added value rests in the Agency’s ability to provide a platform for practitioners to network and exchange best practices on JHA issues at the EU-level. These findings are consistent with those identified in the previous evaluation. Most stakeholders (including CNU/NCPs, FWPs and training participants) that responded to the survey agreed that CEPOL has played an important role as a training provider at the EU-level, particularly in identifying EU training priorities.

Stakeholders consulted throughout the evaluation agreed that CEPOL’s contributions to the EMPACT activities were beneficial in that CEPOL provides tailored training to stakeholders’ needs. This was also evidenced by the document review presented. Additionally, public consultation respondents generally agreed that CEPOL had provided EU added value in developing, implementing and supporting training across the EMPACT crime priorities in the period between 2015 and 2020. It was also pointed out that CEPOL was a valuable partner in developing cross-disciplinary training within specific EMPACT priorities. Some noted CEPOL’s added value in implementing capacity building activities in third countries, which reportedly complement those offered by other EU agencies. Stakeholders consulted during the evaluationevaluation valued CEPOL’s role in coordinating the training offer at the EU level and underlined that CEPOL effectively communicated with the full range of stakeholders, although the evaluation found that CEPOL’s coordinating role was somewhat unstructured and occurred on an ad-hoc rather than planned basis, as mentioned above. Finally, the evaluation found that CEPOL has added some value to the debate about law enforcement training in Europol, though its contribution is difficult to assess. This evaluation has considered to what extent CEPOL’s achievements would have been delivered if another actor or organisation had been responsible for delivering the training. Drawing from the analysis presented above, the evaluation finds that CEPOL is uniquely placed to provide training to law enforcement officials at the EU-level compared to other EU-level agencies or Member State training providers. When stakeholders were asked to provide examples of initiatives or organisations – other than CEPOL – providing training and capacity building for law enforcement at EU and national levels, the most cited organisations included Frontex, Europol and national training organisations. It was suggested that CEPOL’s added value – compared to other EU-level and national initiatives or organisations - rests in its ability to:

·Provide useful training on cross-border issues with an EU lens, which are relevant to a range of law enforcement officials;

·Reach a wider audience of law enforcement officials than could be reached by any other EU Agency or national training academy, specifically for targeted training on cybersecurity, intellectual property infringement or fundamental rights;

·Provide a platform for both national law enforcement officials as well as EU agencies to network and share best practices related to cross-border issues;

·Act as a direct interlocutor between the EU and third countries and international organisations.

The evaluation found, drawing from consultations with key stakeholders, that discontinuing CEPOL’s activities would have a significant negative impact at EU and national levels.

The research team concludes that CEPOL adds value, as an EU agency dedicated to training law enforcement on cross-border issues. Stakeholders consulted throughout the evaluation unquestionably agreed that law enforcement officers in the EU need some training coordinated or provided at the EU-level. This view was also supported by results from the public consultation. Whilst the present evaluation cannot assess the impact of discontinuing CEPOL (as an independent Agency dedicated to law enforcement training), the evaluation asserts that discontinuing EU-level training for law enforcement (as an EU offer) would have a significant impact at EU and national levels.

6.    Conclusions

The overall conclusion of the evaluation, carried out between November 2020 and June 2021, is that the activities carried out by CEPOL are relevant, coherent with EU needs and challenges and bring EU added value. CEPOL is considered as a reliable partner and its knowledge management function is one of its most valued assets. This equally applies to the Agency’s capacity building projects in third countries, which suffer from a low recognition rate even among some stakeholders, but are highly valued by the beneficiaries and those familiar with them.

Yet, the evaluation identified a few issues under the effectiveness and the efficiency criteria. Albeit recognising the Agency’s efforts in terms of outreach, the evaluation found that the population coverage is still extremely limited and the potential to expand it – notably, through train-the-trainers programmes and cascade effect – is not fully fulfilled. Further development of the online activities can contribute to increasing the number of participants; yet, it emerged from the evaluation that this can have some drawbacks with regard to the quality of the experience and its benefits, considering the importance attached to networking and personal relations, regarded as essential to enhance police cooperation. In the face of good synergies and cooperation with most of the other JHA actors including the EU Agencies, coordination is still ad hoc and often unstructured, with CEPOL themselves lacking the authority to fulfil this function. Despite the Agency having developed good quality standards, these are sometimes endangered by the lack of control mechanisms concerning the outsourced activities. In this context, the evaluation found that participants not always having the right profile is an issue and is not really tackled. Some concerns also appear in the governance area, in that the Management Board is not very active and suffers from representation issues. Also, the relocation to Budapest has apparently negatively impacted on the Agency’s corporate culture and the Agency is struggling to attract and recruit qualified and diverse staff.

It would be unfair, though, to conclude that the evaluation pointed to negative conclusions, in the face of the undeniable efforts demonstrated by the Agency and its achievements regards all five evaluation criteria. The evaluation outcomes rather suggest that some of the above critical areas are somehow structural, linked to the sheer size and lack of ‘critical mass’ of CEPOL. Consideration should be given, to reflect upon which format the EU institutions wish to design to continue the relevant activities carried out by CEPOL, with a view to expanding the outreach of the training area of law enforcement, while fully exploting all possible synergies and economies of scale.

Below is a list of the main conclusions of the evaluation.

CEPOL’s key programming documents consider and reflect the objectives of its legal basis and the priorities of the EU Agenda on Security.

This is corroborated by the views of stakeholders that overwhelmingly perceived that CEPOL’s actions and activities over the 2015 to 2020 period have duly reflected the objectives laid out in its mandate.

CEPOL’s objectives are relevant to the EU needs and challenges defined by key strategic documents and CEPOL’s training offer is relevant to the needs of EU Member States.

Training participants across the 27 EU Member States also indicated that over the 2015 to 2020 period CEPOL’s training offer adequately addressed the needs and expectations of their countries.

There is scope for CEPOL to better reach its very large target audience by further investment in the ‘cascade effect’ and continuing to develop its online offer.

Article 2 of Regulation 2015/2219 significantly broadened CEPOL’s target audience to include prosecutors, customs officials, and other persons involved in law enforcement (e.g. forensic lab personnel), though no quantifiable targets were established. The evaluation acknowledge that, given CEPOL’s relatively small size (69 staff members as at 2019) and annual budget (€9.3m in 2019), it cannot be expected that CEPOL directly reaches a sizeable proportion of the target audience through residential training activities. In its current form, cascading relies heavily on the actions of Member States and is not consistently monitored by CEPOL. Also, CEPOL’s use of the train-the-trainer delivery approach is limited (delivering only five sessions in 2019).

CEPOL’s investment in upgrading and expanding its online training offer emerged as a positive strategy for extending its outreach; however, its online delivery tools fell behind those of other EU agencies. The new proprietary online training platform, Law Enforcement Education (LEEd) represents a vast improvement from CEPOL’s previous platform ‘e-Net’.

The yearly development cycle for CEPOL’s training activities is deemed to be too short, jeopardising quality.

Prior to CEPOL’s development of training, the Management Board is required to make decisions about topics, guidelines are drafted, and FWPs undergo selection procedures. Once these processes are complete, according to some stakeholders, there is little time left for quality content development.

Training activities that have been outsourced to CEPOL’s Framework Partners for development and delivery are less exposed to CEPOL’s quality assurance mechanism, resulting in training activities of variable quality.

CEPOL has a quality assurance mechanism in place to control its training activities from development to delivery. Training activities outsourced to FWPs, while still appreciated by participants, suffer from instances of variable quality that escape CEPOL’s quality assurance mechanism.

Participants selected for CEPOL’s residential training do not always have a suitable profile in terms of level of expertise and seniority.

Due to the decentralised nature of CEPOL’s training delivery, CEPOL has limited ability to control the suitability of attendees, e.g. by their rank or profile for the training. Activity Managers within CEPOL have the power to vet and decline participants’ applications, but the evaluation finds that this power is hardly ever used, as it is an established practice to accept the first candidate presented by Member States, regardless of his/her profile.

CEPOL’s Cybercrime Academy is perceived to be an important step taken by the Agency to address emerging needs, however there is room for continued growth in this area.

The establishment of CEPOL’s Cybercrime Academy received mixed reports from stakeholders. While covering a very relevant topic and where demand is high, questions were raised on the actual mission, its limited size and the ability to attract experts to deliver trainings with relevant knowledge and skills. The activities delivered through the Cybercrime Academy have also been hampered by COVID-19, precluding on-site residential activities using the specialised facilities.

CEPOL activities are widely recognised for building trust and facilitating the development of cooperation among law enforcement services.

The evaluation finds that CEPOL’s objectives and tasks helped to improve the ability of Member States to train their law enforcement officials compared to what they could have done at the national level. Moreover, stakeholders confirm that CEPOL’s activities have facilitated an increase in the number of opportunities for cooperation and in the quality of cooperation between national law enforcement services at EU level as well as, to a smaller extent, on a bilateral and regional basis. The majority of training participants agree that CEPOL’s training activities provide concrete opportunities to establish networking with their European counterparts and that CEPOL adds unique value to develop cooperation that would not have otherwise been achieved through national means. This outcome is particularly attributed to CEPOL’s exchange programme and residential activities.

CEPOL’s work in third countries is highly valued by informed stakeholders, though some within CEPOL’s governance lack awareness of these activities.

CEPOL projects in third countries effectively contribute towards improving the stability in the EU neighbourhood and in building trust with neighbouring countries, however, this branch of activities suffers from low recognition within CEPOL’s own governance. Training in third countries is a crowded market – international organisations, non-EU countries, EU Member States and other Commission funded projects all aim for the same target audience. Among these, CEPOL is praised unanimously for its commitment to provide training activities responding to the needs of partner third countries. As per mandate, CEPOL manages dedicated Union External Assistance funds to fulfil its objectives in third countries.

CEPOL’s governance

While there is a good relationship between the CEPOL Executive Director and the Management Board, Management Board meetings are hampered by representation issues, low engagement by some Members, and the disproportionate focus on administrative matters.

The CEPOL Management Board suffers from representation issues, such as low engagement of several Member States; gender imbalance; no customs or border officials; the levels of seniority of representatives are varied. While most Management Board decisions are related to content and strategy, the agenda of Board meetings is cramped with administrative items, leaving little room for strategic discussion.

The relocation of CEPOL and the consistent turnover of staff has resulted in a shift in corporate culture and there are challenges in attracting (appropriately qualified) staff.

The number of CEPOL staff increased steadily over the period 2015 to 2019 (from 45 to 69), though there has been high staff turnover. The salary correction coefficient applied to Budapest indeed plays a key factor in CEPOL’s difficulty in attracting and retaining staff. CEPOL lacks an inspiring corporate culture that motivates staff to commit to the agency and its mission. Once perceived as a community, CEPOL is now perceived as a transitory workplace where it is hard to invest in social commitments.

CEPOL’s remit and activities appear to be coherent with and complementary to other relevant actors at the EU level.

JHA Agency stakeholders were generally positive about their relationship with CEPOL; yet, the extent of coordination is ad hoc, varies by Agency and CEPOL’s engagement with JHA agencies and EU institutions. It emerged that CEPOL could adapt its working approach to maximise potential synergies. No structured methodology has replaced the JHA Training Matrix previously used; instead, CEPOL undertakes coordination through a peer-to-peer approach.

Annex I: Procedural information

The external evaluation of the European Union Agency for Law Enforcement Training (CEPOL) was carried out in the period December 2020 - June 2021. Preparatory steps, such as the Terms of Reference for hiring a contractor were initiated in Spring 2020 and the roadmap of this initiative was published on 1st July 2020. The Management Board was informed about the upcoming evaluation at its meeting of December 2017. The agenda planning reference for this evaluation is PLAN/2020/8322.

An inter-service steering group was set up for the evaluation. The following Directorates General (DGs) and services followed the invitation to nominate a participant in the interservice steering group: the Secretariat-General of the Commission (SG), the Legal Service (LS), the European Anti-Fraud Office (OLAF), DG Human Resources and Security (HR), DG Budget (BUDG), the Internal Audit Service (IAS), DG Taxation and Customs Union (TAXUD), DG Justice and Consumers (JUST), DG International Partnerships (INTPA), DG Neighbourhood and Enlargement Negotiations (NEAR), the Service for Foreign Policy Instruments (FPI) and the European External Action Service (EEAS). DG Migration and Home Affairs (DG HOME) chaired the meetings of the inter-service steering group, which was systemically consulted during the evaluation process, including on the Terms of Reference in September 2020. The kick-off meeting with the contractor took place on 3rd December 2020. The questionnaire for the public consultation, for which the contractor prepared a first draft, was sent to the inter-service steering group for input in December 2020. The inter-service steering group met in January, to discuss the draft inception report; in March, to discuss the draft interim report; in May, to discuss the draft final report; all meetings took place in the presence of the contractor. Before, during and after those meetings, the inter-service steering group, had the opportunity to provide their comments on the different draft reports, ask for clarifications and submit their suggestions.

The services were requested within framework contract HOME/2015/EVAL/02. Three possible contractors applied for carrying out the evaluation. The chosen contractor was Ernst & Young (E&Y) who collaborated with RAND Europe in carrying out the external evaluation. The final report of the external evaluation was submitted to the Commission for acceptance on 11th June 2021.

Annex II: Stakeholder consultation

A broad stakeholder consultation accompanied the external evaluation. The aim of the process was to obtain views on the five evaluation criteria, on the functioning of the Agency and on possible future challenges and developments from a range of stakeholders working with the Agency or benefitting from the work of the Agency. The contractor carried out the targeted stakeholder consultation. The Commission's standards for Stakeholder Consultations were duly taken into account and met. As part of the stakeholder consultation, the Commission conducted a public consultation, with the purpose of gathering views from private individuals, organisations, the industry and the public administration. The replies to the stakeholder consultation addressed all evaluation questions. The outcomes of the interviews, the targeted surveys and the public consultation are reflected fully in the external study and in this Staff Working Document. Summarising them in this Annex would be a repetition of the conclusions of the external evaluation set out in the main text of this document.

1.Public consultation

In addition to the targeted consultations, the Commission organised an internet-based public consultation. The public consultation was launched on 18 March 2021 on the European Commission's website and was open for 8 weeks. The questionnaire was available in English, French and German and inputs could be submitted in all official languages of the EU. The aim of this consultation targeting the general public was to gather views on the Agency. The consultation covered the majority of the evaluation criteria: effectiveness, relevance, coherence and EU value added. As such, the public consultation was intended to form part of the input for the external evaluation.

Profile of respondents

In total, 26 responses were received. Of the 26 contributions received, 14 (54%) were submitted on behalf of EU citizens. Public authorities were the next largest group, accounting for seven (27%) of responses, followed by academic/research institutions (n=2, 8%), non-governmental organisations (NGO) (n=1, 4%), trade union (n=1, 4%) and other (n=1, 4%). Among the seven contributing public authorities, four indicated international contribution, two regional, and one national scope of their contribution. With respect to the geographical distribution of respondents, all contributions came from EU Member States. By far the largest group of contributions (n=10, 38%) was submitted by respondents based in Germany, followed by Hungary (n=4, 15%) and Belgium (n=3, 12%). Another eight Member States were responsible for the remainder of contributions, no responses were received from 16 Member States. The vast majority of respondents (n=22, 88%) reported actively working in the field of law enforcement training or related domains, while only three (12%) respondents reported being interested in the field without having actively worked in it. Respondents’ background was then likely reflected in their high levels of familiarity with CEPOL. Relatedly, the vast majority of respondents (n=22, 85%) reported some prior form of engagement with CEPOL. About a quarter of respondents (n=7, 27%) reported having cooperated with CEPOL in the past, slightly fewer (n=5, 19%) reported having participated in CEPOL training in the past and the largest group (n=10, 38%) reported having done both. Lastly, 20 (77%) of respondents reported being a registered user in LEEd (CEPOL’s e-learning platform).

Responses to closed substantive questions pertaining to the four evaluation criteria formed the basis for this analysis. For each question, cross-tabulations and other descriptive statistics were prepared to assess the frequency of responses and the prevalence of particular views. Where appropriate, quantitative data were broken down by various demographic categories uncovered in the first stage of the analysis. The consultation questionnaire contained seven open-text questions. The first six invited respondents to elaborate on topics raised in the previous set of questions, responses to these open-ended questions were summarized in a narrative form complementing the analysis of the preceding related questions. Question 60 offered respondents the option to add any closing comments. These comments were summarized in a narrative form in a standalone section. Below is a summary of the main findings from the public consultation; all detailed result by evaluation criterion are providedin the evaluation.

Results

In terms of effectiveness, overall, most respondents agreed that CEPOL has met all the listed objectives, with the exception of providing training to law enforcement trainers, where only 10 (42%) respondents agreed that CEPOL had met the objective. The two objectives most likely to be viewed by respondents as met by CEPOL were (1) supporting Member States training on the implementation and use of international and Union instruments and (2) developing, implementing, and supporting training on specific thematic areas (both endorsed by 22 (92%) of respondents).

Concerning relevance, all respondents agreed that law enforcement officers in the EU needed some training coordinated at the EU level. Most respondents agreed that CEPOL had responded to the needs of law enforcement officials in the EU as well as to the needs of EU agencies and institutions over the reference period of 2015-2020.

Looking at coherence, contributors held generally positive views of CEPOL’s coherence and cooperation with other ongoing efforts. The majority of respondents (n=15, 65%) agreed that CEPOL’s activities were synergistic with the work of other EU agencies and institutions. Respondents were more likely to disagree (n=10, 42%) than agree (n=4, 17%) that CEPOL’s activities overlapped with or duplicated the work of other EU agencies and institutions. Further, most respondents (n=13, 57%) agreed that CEPOL’s activities incorporated focus on human rights and were more likely to agree (n=10, 42%) than disagree (n=1, 4%) that CEPOL’s activities in non-EU countries were coherent with other EU cooperation mechanisms with third countries.

Referring to added value, the results are consistent with respondents’ perception of CEPOL’s effectiveness. Areas where respondents were most likely to agree CEPOL provided added value were training on specific thematic areas (n=22, 92%), development and exchange of best training practices (n=20, 83%), and developing, upgrading, and evaluating learning tools and methodologies (n=20, 83%). By contrast, only half of respondents (n=12, 50%) agreed that CEPOL added value in providing training to law enforcement trainers and twelve (52%) agreed that CEPOL added value in providing training in preparation for EU missions.

In addition to the open-text comments, respondents had the option to upload a document to further support their contribution to the consultation. Two supporting files were received. One document stressed the importance of improving training content on victims’ rights and victimisation and of enhancing methods of communication with victims of crime among law enforcement agencies. The second submitted file was a position paper offering suggestions echoing points made in open-text responses throughout the survey and discussed above in this report, including: (1) developing a more harmonized European police training standard, (2) utilising CEPOL to enhance cooperation and exchange between police officers, (3) expanding in-person training provision, and (4) reconsidering CEPOL’s location.

2.Online surveys

Four online surveys were launched to collect (comparable) information from a large number of stakeholders, and to gather specific contributions that would not be feasible to obtain only from interviews. Input from the survey contributed both to answer the evaluation questions and to feed the case studies.

The online surveys were targeted at the following categories of stakeholders:

·Survey 1: CEPOL National Units (CNUs) and National Contact Points (NCPs).

·Survey 2: Member State representatives from police academies and training institutes that are framework partners of CEPOL (e.g. Security Academy of the Federal Ministry of Interior in Austria, Cyprus Police, Estonian Academy of Security Sciences).

·Survey 3: Representatives from third countries with cooperation agreements with CEPOL to understand the external dimension of CEPOL.

·Survey 4: A sample of CEPOL training participants/target audiences.

The four surveys were launched at different dates, to allow the research team to collect contact details from external stakeholders before proceeding with the launch. Initially planned to stay open for three weeks, it was agreed with the contractor to extend the deadline, in order to collect input from all countries. All EU Member States participating in CEPOL (i.e. excluding Denmark) were covered, as well as third countries with a cooperation agreement with CEPOL. The number (and rate) of responses collected is presented in Table 19.

Table 19: Rate of response per category of stakeholder

3.Stakeholder interviews

Interviews were conducted with stakeholders from 41 organisations across four stakeholder groups and three experts, as listed in Table 35 . Topic guides and the content of interviews were tailored to the expertise of the interviewee. Stakeholders were identified through suggestions from the Commission, CEPOL and a stakeholder mapping process. The key information obtained was coded into an evidence grid, which served to capture the relevant information by each evaluation question.

Table 20: Organisations consulted for interviews

Annex III: Methods and analytical models

1.Documentary review

The purpose of this task was to review and collect data and information from all available sources to canvass CEPOL’s objectives and tasks, management and governance, internal working methods and procedures, and activities. The document review was structured to feed answers to all the evaluation questions. In addition to canvassing the CEPOL website for relevant documentation, CEPOL and the Commission shared a large number of documents that the evaluation should consider. More details on the documentary sources can be found in the evaluation. A data extraction template was prepared to structure the review of documentation and make sure that information for the evaluation is collected in a systematic and efficient way. A simplified version of the template is provided in the table below. In addition to these high-level categories, information was also extracted and organised according to the indicators specified in the evaluation grids.

Table 21: Simplified version of the fields included in the data extraction template

2.Public consultation, online surveys, stakeholder interviews

See Annex II for a synopsis of the consultation activities including the public consultation, online surveys and stakeholder interviews.

3.Expert panel

The research team was supported by three international experts on law enforcement and crime-related matters in the EU. At the outset of the evaluation, the consultant team benefitted from the experts’ thoughts, comments and suggestions on:

·The basic parameters of the evaluation;

·The issues at stake and how CEPOL fits within the broader EU JHA landscape;

·Our approach for the stakeholder and expert consultation, and which organisations and individuals to approach.

The experts provided feedback on all drafts and deliverables prepared over the course of this evaluation. The experts helped steer the overall approach and made suggestions regarding literature, stakeholders and methodologies. The members of the expert panel are listed in the table below.

Table 22: Members of the expert panel

4.Virtual visits to CEPOL

On Wednesday 16 and Friday 18 December 2020, in the preparatory phase of the evaluation, the research team held a series of virtual (because of travel restrictions due to the COVID-19 pandemic) visits or meetings with senior representatives from CEPOL. This included CEPOL’s Executive Director, Mr Detlef Schroeder, as well as the relevant Unit Heads from Corporate Services and Operations (specifically, Training and Research and International Cooperation). These meetings allowed the research team to collect preliminary contextual information about the function of CEPOL to help guide the evaluation. On Monday 12 and Tuesday 13 April 2021, in the analysis phase of the evaluation, a second series of virtual visits or meetings with senior CEPOL staff were held. These meetings allowed the research team to ask CEPOL more targeted questions, stemming from the analysis of the data collected to date and test preliminary recommendations and policy ideas. All of these conversations were guided by a topic guide, audio recorded, and written notes shared among the research team.

During the first virtual visit to CEPOL, CEPOL’s staff provided the research team with detailed information about the agency’s inner workings and factors influencing CEPOL’s operations since the introduction of the new legal basis in 2016. During the second virtual visit to CEPOL, the research team and CEPOL staff discussed some of the key findings that emerged from the evaluation’s other data collection activities.

5.Case studies

Three case studies were conducted focusing on:

·CEPOL’s governance, human resources management and working methods, aimed at looking at the high-level oversight and steering of CEPOL according to its governing arrangements (the Management Board, Executive Director and Senior leadership), how CEPOL attracts, retains and develops staff and wider human resources and decision-making processes and accountability processes. The case study provides inputs relevant to answer evaluation questions 10; 14, 16 17 and 21.

·The external action of CEPOL (intra and extra-EU), aimed at investigating CEPOL’s international cooperation and capacity building activities in Third Countries, taking a closer look at the extent to which, and with what results, CEPOL provides training to prepare law enforcement officials for participation in Union missions and training for law enforcement officials from third countries.

·Lessons learned from other EU training units (e.g. within Europol, EASO, Frontex) who deliver training to law enforcement officials and cross-professional audiences across the EU, to identify lessons learnt and transferrable elements for CEPOL.

The aim of the case studies was to analyse more in depth some key elements linked to CEPOL’s structure, governance and activities. The information collected through the case studies was used for the purposes of the overall evaluation, including answering specific evaluation questions, but also to provide additional details on the above-mentioned topics, providing a detailed picture of the three key elements selected, which would have not been addressed as comprehensively in the overall evaluation. Each case study was carried out using a combination of desk and field research. Data collection benefitted from the evidence gathered from interviews and online surveys implemented targeting the whole evaluation. Additional, ad-hoc interviews were organised for each case study to collect new and more specific information and data.

6.Workshops

On Thursday 15 April 2021 the research team held a virtual workshop with representatives from the Commission to discuss some key themes emergent from the evaluation and potential recommendations. In the meeting, the research team presented the draft conclusions and corresponding recommendations and invited questions from the Commission to ensure the recommendations have been fully assessed in terms of their legal, practical, political and economic feasibility. Discussion was also aimed at ensuring that the proposed stakeholders for each recommendation are correct.

On Wednesday 12 May and Monday 17 May the research team held two virtual workshops with representatives from CEPOL to discuss key findings and potential recommendations. Prior to these workshops CEPOL staff were provided with a written summary of the key issues and suggestions. During the sessions the research team presented the key findings and suggestion, and this was followed by detailed discussions with members of the CEPOL staff to ensure issues were fully understood and to check feasibility of the suggestions being made.

Detailed minutes of all workshop were recorded and used for refinement of and integration with the final report.

The two final workshops - held with representatives from the Commission and CEPOL respectively – aimed to discuss some of the emerging findings from the evaluation and preliminary suggestions to address identified areas for improvement. These issues are listed below, along with a summary of the feedback received from the two stakeholder groups.

7.Limitations of evaluation and mitigation measures

There are a number of limitations of the data collected to inform the evaluation, which should be borne in mind when interpreting the findings. Below we summarise the limitations and mitigation measures taken by the research team to address them.

1.Measuring effectiveness: Measurement of the Agency’s effectiveness is limited by lack of statistical, financial, and administrative data to test, challenge and validate the expert judgements and stakeholder opinions collected through fieldwork activities. For example, there is an absence of data to assess whether the skills and knowledge of CEPOL training participants have actually increased as a result of the training provided.

2.Relying on self-reported data: Data collected from stakeholders through interviews and surveys is self-reported, may be subject to recall bias and may reflect the specific agenda of those consulted.

Mitigation measure to points 1 and 2: Triangulation of self-reported data with data analysed from documentary sources including factual information on the functioning of CEPOL, information on CEPOL’s activities, previous evaluation documents, and documents related to CEPOL’s partnerships.

3.Attributing outcomes: In the absence of a counterfactual, there are challenges in unequivocally attributing outcomes to the activities of CEPOL. The aim of this evaluation was to assess the strength of evidence for CEPOL’s progress against the intended intervention logic, rather than assess direct, verifiable causality based on strong empirical evidence.

a.Mitigation measure: The evaluation is a contribution analysis, rather than an impact evaluation assessing causality.

4.Quantifying benefits: The inability to quantify the benefits of CEPOL means that this analysis is unable to make any concrete conclusions about the cost-effectiveness of the Agency.

a.Mitigation measure: The efficiency analysis focuses on the costs of CEPOL’s activities and the efficiency of the Agency in light of the resources available.

5.Representativeness of stakeholder views: There may be selection bias because those interviewed and surveyed were selected by the research team following a stakeholder mapping exercise (involving the Commission and CEPOL) and may not be representative of all stakeholders of CEPOL.

a.Mitigation measure: Selection of stakeholders across a wide range of groups including those at the EU, Member State, and international levels.

Annex IV: Intervention logic

Annex V: Evaluation questions

Annex VI: Observations and Recommendations by the Management Board

European Union Agency for Law Enforcement Training

Five-Year Evaluation Report

2015-2020

Observations and Recommendations by the Management Board

Table of Contents

Table of Contents

Foreword

Area evaluated: Effectiveness

(extend outreach of target audience through a cascading effect)

Area evaluated: Effectiveness (participant’s profile)

Area evaluated: Efficiency (MB decision process, planning and monitoring procedures)

Area evaluated: Efficiency (staff turnover)

Area evaluated: Efficiency (quality assurance mechanism for outsourced training activities)

Area evaluated: Efficiency (Cybercrime Academy)

Area evaluated: Relevance

Area evaluated: Coherence (strengthen cooperation with other JHA and CEPOL’s role as a coordinator on training aspects)

Area evaluated: Added value (including training activities in third countries)

Other aspects

Observations and Recommendations by the Management Board

Foreword 

Article 32 of the European Parliament and Council Regulation (EU) No 2015/2219 70 stipulates that:

‘By 1 July 2021 and every five years thereafter, the Commission shall ensure that an evaluation assessing, in particular, the impact, effectiveness and efficiency of CEPOL and of its working practices is carried out. The Commission shall submit the evaluation report to the Management Board. The Management Board shall provide its observations on the evaluation report within one month from the date of receipt. The Commission shall then submit the final evaluation report, together with the Commission's conclusions, and the Management Board's observations in an Annex thereto, to the European Parliament, the Council and the Management Board. The findings of that evaluation report shall be made public’.

In order to support the Management Board to provide its observations on the evaluation report, a reporting panel was established by Decision of the Management Board 33/2021/MB.

The Management Board concurs with the evaluator’s report and main recommendations. Yet some of the recommendations cannot be effectively addressed with the current limited human and financial resources allocated to CEPOL.

The Management Board reviewed the five-year evaluation for each of the areas covered and provides the following observations and recommendations.

Area evaluated: Effectiveness

(extend outreach of target audience through a cascading effect)

Observations

The assessment of CEPOL’s effectiveness was positive as the evaluation found that CEPOL’s key programming documents consider and reflect the objectives of its legal basis and the priorities as set out in the 2015 EU Agenda on Security. Moreover, the introduction of the EU-Strategic Training Needs Assessment methodology, developed to identify gaps in knowledge, skills and competencies and training needs, has strongly improved the prioritisation of training needs.

Yet the study recommends that CEPOL should aim to reinforce the outreach and impact of the Agency. This could be achieved, for instance, by integrating cascading (i.e. multiplying the outreach as a result from the training of trainers and peer-to-peer transfer of knowledge) more systematically into all activities, stepping up the train-the-trainer programme, with a special focus on law enforcement senior managers, and continuing to develop and strengthen its online offer. This would help CEPOL to better address its target audience, as broadened in the legal basis in force since 2016, now including prosecutors, customs officials, and other persons involved in law enforcement (e.g. forensic lab personnel).

MB reflections

The Management Board, while acknowledging that cascading remains a national responsibility, encourages CEPOL to do the utmost in its capacity to further extend outreach of the target audience.

MB recommendations

1.CEPOL shall further expand the train-the-trainer programme, and a special focus on law enforcement senior managers.

2.CEPOL shall investigate the options for professional certification that could be offered in the context of train-the-trainer programme versus applicable standards on the market (e.g. ISO 29993:2017 Trainer Certification)

3.CNUs supported by CEPOL shall develop a national cascading system to be systematically integrated into the training activities and monitored in terms of outreach to the broaden target audience, with due regard to the specificities at national level.

4.CEPOL should further continue the successful implementation of online qualification services.

Area evaluated: Effectiveness (participant’s profile)

Observations

The evaluator identified that participants selected for CEPOL’s residential training do not always have a suitable profile in terms of level of expertise and seniority. However, stakeholders reported that CEPOL Activity Managers are now reviewing more carefully participant applications.

MB reflections

Management Board takes note that the issue is being addressed and encourages both CNUs and CEPOL’ Activity Managers to continue the practice of conducting selection of participant in strict compliance with the set criteria.

Area evaluated: Efficiency (MB decision process, planning and monitoring procedures)

Observations

The Management Board takes note that in terms of efficiency, the evaluation found that the benefits stemming from participating in CEPOL’s activities outweighed the costs for Member States, and that CEPOL’s resources were managed well.

The evaluation also concluded that there were some inefficiencies related to the Management Board’s decision-making process, CEPOL’s planning processes and monitoring procedures. Board meetings are attended by large numbers of delegates per Member State, not always properly representing the full spectrum of the law enforcement bodies and their inputs. Several stakeholders observed that, while most Management Board decisions are related to content and strategy, the agenda of Board meetings is cramped with administrative items, leaving relatively little room for strategic discussion.

It was recommended that Member State representatives further strengthen coordination within their countries, with a view to gather and address the needs of the entire law enforcement community. In particular, the evaluation highlighted the importance of the role of CEPOL’s Management Board, as a body which should reflect and relay all inputs and demands beyond the academic world, notably from the operational level in their Member States, to be fed into the Board’s strategic decisions. In this context, attention should also be paid to gender representation within the Management Board, as also highlighted by the European Parliament, which – in the context of the 2019 discharge procedure – urged CEPOL to ensure gender balance when nominating their members.

MB reflections

The Management Board agrees with the recommendation that the Board meetings should focus more on strategic discussion and less on administrative matters. At the same time, the MB would like to emphasise the importance of the Operational and EU Strategic Needs Assessment as an essential tool to reflect the operational demands and bring up needs of the field, hence the commitment of Member States in participating to these assessments is crucial.

On participation of other law enforcement bodies in the MB meetings, the MB highlights that police usually represents the majority within the law-enforcement community, therefore the representation level mirrors the real situation. The same applies to the matter of gender representations that reflects the national gender balance within MS, therefore both aspects are beyond CEPOL’s outreach.

MB recommendation

5.The MB shall further streamline the meeting agenda to focus on strategic matters.

Area evaluated: Efficiency (staff turnover)

Observations

While the evaluator assessed that CEPOL’s internal structure appears to operate efficiently, it was noted that CEPOL’s staff turnover might have affected the Agency’s overall efficiency during the 2015 to 2020 period. The turnover of staff has resulted in a shift in corporate culture and challenges in attracting appropriately qualified staff.

MB reflection

The Management Board takes note of the finding and highlights that the relocation of CEPOL from UK to Hungary had a strong impact on staff turnover due to lower salary coefficient combined with under graded positions for key functions in the establishment plan.

MB recommendation

6.The MB hereby takes this opportunity to flag these aspects for the stakeholders in the decision-making fora, and recommends that the grading of posts in CEPOL should be aligned with those in other JHA Agencies.

Area evaluated: Efficiency (quality assurance mechanism for outsourced training activities)

Observations

The evaluator noted that CEPOL now operates under a hybrid business model, which combines the traditional grant approach, consisting in tendered topics among the eligible Framework Partners, and the ‘new’ CEPOL Knowledge Centre method, consisting in a cluster of experts specialised in a topic. This mixed method is meant to streamline and simplify the grants management and has been

put in place after the Management Board rejected the complete abolition of the grant system. CEPOL was generally capable of absorbing the EU funding it received.

According to the evaluators’ finding, the yearly development cycle for CEPOL’s training activities is deemed to be too short, jeopardising quality. Prior to CEPOL’s development of training, the Management Board is required to make decisions about topics, guidelines are drafted, and Framework Partners undergo selection procedures. Once these processes are complete, according to some stakeholders, there is little time left for quality content development, as the yearly cycle has come to an end. Consequently, Framework Partners with no ‘off-the-shelf’ training solutions that could be offered during this limited timeframe are discouraged from applying to organise a course. The result is that the number of grant applications from Framework Partners is constantly decreasing.

Among its key finding, the evaluator notes that the training activities that have been outsourced to CEPOL’s Framework Partners for development and delivery are less exposed to CEPOL’s quality assurance mechanism. This results in training activities of variable quality. CEPOL has a quality assurance mechanism in place to control its training activities from development to delivery, which was found to be effective in guaranteeing high-quality content and trainers when CEPOL is responsible for both developing and delivering training. Training activities outsourced to Framework Partners, while still appreciated by participants, are sometimes of variable quality, as they have not always been systematically screened by the CEPOL’s quality assurance mechanism.

MB reflection

The Management Board shares the opinion that CEPOL has proven to be successful in implementing the funds it received, however considers that using the term ‘absorption of EU funds’ is not appropriate in the context of EU Agencies.

Management Board recognises the establishment of CEPOL Knowledge Centre as an efficient model of implementation of training activities, where the Agency has taken on board the central administration of the budget. However, the additional necessary human resources are missing to deploy this model on a full scale, for all training activities.

 MB recommendation

7.CEPOL should do the utmost in its capacity to gradually extend the CEPOL Knowledge Centre model to other training activities, within the limits of available resources.

Area evaluated: Efficiency (Cybercrime Academy)

Observations

According to the evaluator, the CEPOL’s Cybercrime Academy is perceived to be an important step taken by the Agency to address emerging needs but should be further improved, especially with regards to the training offers addressing specific technical skills, such as cryptocurrency, access to (big) data and encrypted data, which can hardly be covered by single Member States alone.

MB reflections

The Management Board shares the apprehension of the evaluator that training offer should focus on topics that cannot be addressed at national level, especially on evolving new technologies such as Artificial Intelligence, big data analysis and data encryption, as evidenced by the training needs assessments conducted by CEPOL. At the same time, MB considers that Cybercrime Academy should also envisage a train-the-trainer component, aiming for qualified trainers.

MB recommendation

8.CEPOL should be enabled to further invest into its cybercrime training offer, to cover for relevant topics to be better addressed at the European level.

9.MB considers in reference to recommendation n°2 that cyber related train-of-trainers activities should be best supported by certification of the trainings.

Area evaluated: Relevance

Observations

The evaluation confirmed that CEPOL’s objectives are relevant to the EU needs and challenges defined by key strategic documents and CEPOL’s training offer is relevant to the needs of EU Member States.

The evaluator also mentions that there were mixed findings in relation to CEPOL’s ability to adapt its training offer to scientific, technological and socio-political developments.

MB reflections

Management Board considers that second remark could have been better substantiated to enable further analysis. Nevertheless, MB agrees that the training

offer should be flexible to adapt to new developments, and to this end CEPOL conducts regular strategic and operational training needs assessments.

MB recommendation

10.CEPOL should consider the further application of the existing model to satisfy emerging training needs.

Area evaluated: Coherence (strengthen cooperation with other JHA and CEPOL’s role as a coordinator on training aspects)

Observations

The evaluation concludes that CEPOL’s remit and activities appear to be coherent with and complementary to other relevant actors at the EU level. Yet, the extent of coordination is ad hoc, varies by Agency and by CEPOL’s level of engagement with JHA agencies and EU institutions. This means that there is some room for improvement.

The evaluation highlights the importance for CEPOL to streamline its activities to focus on key priorities within the European Multidisciplinary Platform Against Criminal Threats, the Security Union Strategy and recent internal security strategies, while remaining open to assessing and catering for the specific training requests of Member States within this framework. In that regard, CEPOL should increase its cooperation with other Agencies in general, and with Europol in particular, in order to maximise synergies as widely as possible. In this context, CEPOL has accepted to take the lead in the coordination of the common horizontal strategic goal on “capacity building through training, networking and innovation” during the upcoming cycle 2022-2025 of the European Multidisciplinary Platform Against Criminal Threats. Likewise, it is recommended CEPOL should increase its coordinating role, also when contributing to implement all the internal security strategies, including the Counter Terrorism Agenda, adopted in December 2020, and both EU Strategies on Organised Crime and on Trafficking, adopted in early 2021.

Linked to this, it is recommended that CEPOL plays a central role in coordinating law enforcement training among all different stakeholders, and notably the other JHA agencies. Rather than being in competition with other EU training stakeholders, CEPOL should aim to become “the” EU hub for law enforcement training. While the Agency is not in a position to cater for all training needs itself, it has a unique role in mapping and monitoring the catalogue of existing law

enforcement training offers provided by all JHA agencies and other stakeholders and to act as a portal for law enforcement practitioners looking for suitable training opportunities at the EU level.

MB reflections

The MB does not fully share the opinion on the coordination matters, e.g referring to the existing working arrangements, but agrees that there is some room for improvement.

The Management Board supports the initiative that CEPOL becomes “the EU hub for law enforcement training”. However, MB considers that the coordinator role cannot be effectively achieved in absence of adequate staffing level and moreover until this is catered for in the CEPOL’s legal mandate which should be complementary to the legal mandates of other JHA Agencies.

Area evaluated: Added value (including training activities in third countries)

Observations

The evaluation concluded that, regarding the EU added value, all consulted stakeholder groups were of the view that CEPOL indeed provides added value with regard to training on specific crime areas compared to what is offered at the national level. CEPOL capacity building projects in third countries effectively contribute towards improving the stability in the EU neighbourhood and in building trust with neighbouring countries. Although training in third countries is a crowded market, CEPOL is praised for its commitment to provide training responding to the needs of partner third countries. As per mandate, CEPOL manages dedicated Union External Assistance funds to fulfil its objectives in third countries.

Finally, as regards the activities carried out by CEPOL in third countries, it is confirmed that these should continue, where possible, and in line with the EU’s external policy. However, considering the resources available to the Agency, it is recommended they should not deflect attention from the core priorities referred to above, while ensuring that they meet concrete needs and have a real added value.

MB reflection

The Management Board agrees with evaluator’s finding and recommendation on activities in third countries.

MB recommendation

11.CEPOL will continue implement training activities in third countries to the possible extent with due consideration to the available capacities of CEPOL and Member States.

Other aspects

The significant changes in the EU internal security landscape generate an increased demand for law enforcement training where the priorities are shifting towards topics such as cybercrime, migration, counterterrorism, artificial intelligence.

In multi-annual perspective, the outreach of CEPOL became more than double: number of participants has grown from ca. 18,000 in 2016 to ca. 40,000 in 2021 which reflects exploding demand and necessity for the training activities that the Agency delivers.

Six years after the entry into force of its new legal mandate 71 on 1 July 2016 and sixteen years since the inception of CEPOL as an EU Agency, CEPOL operates with 33 Temporary Agents in its establishment plan. 

Due to insufficient resources, this demand could be only followed by increasing the online learning component, which did not entirely satisfy the need of the stakeholders. The number of tasks assigned to the Agency and its stakeholder expectations continue to grow, which CEPOL cannot satisfy due to the lack of available financial and human resources.

The present mandate brought about new coordination and analytical tasks for CEPOL, together with implementing capacity-building projects in third countries by managing dedicated Union External Assistance funds, however this was not accompanied with any additional training agents post.  

The MB is of the opinion that genuinely addressing most deficiencies identified by the evaluation report and confirmed by the reflections of the Management Board, just as some of the recommendations (notably MB recommendations 3, 6, 7 and 8) emerging from the evaluation report, would require amendment of CEPOL’s legal mandate and the related Legal and Financial Statement, to provide the agency with additional resources, justified by evidence-based, clear business needs.

Accordingly, MB recommends for the consideration of the European Commission to table a legislative proposal on amending CEPOL’s current legal mandate and the related Legal and Financial Statement, in order to increase the Agency’s impact and address the current recommendations.

(1)    COM(2013) 172 final.

(2)     https://www.consilium.europa.eu/en/policies/eu-fight-against-crime/ and https://ec.europa.eu/home-affairs/what-we-do/policies/law-enforcement-cooperation/operational-cooperation/empact_en   https://www.consilium.europa.eu/en/policies/eu-fight-against-crime/ and

(3)    European Commission. n.d. “Better regulation guidelines - Evaluation and fitness checks.” Available at: https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines-evaluation-fitness-checks.pdf

(4)     http://ec.europa.eu/smart-regulation/guidelines/tool_42_en.htm

(5)    The intervention logic describes how an intervention is expected to attain its global objectives, by listing all its activities/inputs (including the governance arrangements for the Agency, the internal structures and procedures and the administrative and financial arrangements) and expected effects (outputs, results and impacts), as well as the assumptions that explain how the activities/inputs will lead to the effects in the context of the intervention. An intervention logic aims to reconstruct the expected chain of events of an intervention by using a model of causality, i.e. demonstrating how an intervention was triggered by needs existing and how it was designed, with the intention of producing the desired changes. European Commission n.d. Better Regulation: Guidelines and toolbox. Accessed 22 April 2021 from:  https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-guidelines-and-toolbox_en

(6)    Article 3, Regulation (EU) 2015/2219. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32015R2219&from=EN

(7)    Adopted in 2009 by the European Council to provide a framework for EU action on issues related to justice and home affairs.

(8)    Prior to the Regulation CEPOL’s target audience were senior police officers in Member States, but this remit was broadened to allow CEPOL to cater to the needs of the wider population of European law enforcement officials (including the judiciary/prosecutors).

(9)    Article 3, Regulation (EU) 2015/2219. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32015R2219&from=EN

(10)    European Commission. 2015. European Agenda on Security: Strengthening EU cooperation in the fight against terrorism, organised crime and cybercrime. Available at: https://ec.europa.eu/home-affairs/what-is-new/news/news/2015/20150428_01_en

(11)    A four-year cycle set up by the EU in order to create a greater measure of continuity for the fight against serious international and organised crime:  https://ec.europa.eu/home-affairs/what-we-do/policies/law-enforcement-cooperation/operational-cooperation/empact_en   https://ec.europa.eu/home-affairs/what-we-do/policies/law-enforcement-cooperation/operational-cooperation/empact_en  

(12)    Article 4.1. Regulation (EU) 2015/2219. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32015R2219&from=EN

(13)    The CEPOL Organogram as of 1 January 2020 describing the changes that will be effective from 2021.

(14)    CEPOL 2020 Single Programming Document Years 2020-2022. Available at https://www.cepol.europa.eu/sites/default/files/Annex-23-2019-MB.pdf p130-131

(15)    In 2015, a €2.5 million project was granted to CEPOL by the Commission to implement a counter-terrorism capacity building project in four countries in the MENA region (CEPOL Annual Report, 2015). The project implementation ended in 2017 and that same year, CEPOL signed two new agreements with different Commission services (DG NEAR and EEAS). This was for the implementation of two capacity-building actions: the Financial Investigation In-Service Training Programme in the Western Balkans (maximum budget of €2.5 million for 2018-2019) and the EU/MENA Counter-Terrorism Training Partnership (maximum budget of 6.4 million for 2018-2020) (CEPOL CAAR 2018). In 2019, CEPOL signed an agreement with DG DEVCO for the implementation of the action ‘CT INFLOW – Enhancing Information Exchange and Criminal Justice Response to Terrorism in the Middle East and North Africa’ (maximum budget of 7.5 million for 2019-2021) (CEPOL CAAR, 2019).

(16)    Includes learning and training activities carried out in third countries.

(17)    CEPOL 2016 Single Programming Document Years 2016-2018.

(18)    According to Eurostat, in 2019 there were 1,481,297 police officers in the EU 26 (Denmark excluded as not part of CEPOL and missing countries estimated with previous available year value).

(19)    Eurostat does not provide data on all people in law enforcement.

(20)    The survey of training participants shows that, on average, participants attended 3 CEPOL activities between 2015 and 2020. Survey of training participants (n=769).

(21)    Data from a provisional version of the CAAR.

(22)    CEPOL reported 1334 total participants from CT2 and FI WB projects, however 108 exchanges were double counted both as FI WB exchanges and CEP exchanges. The table reports data without the double counting.

(23)    No disaggregated data available for 2019.

(24)    Eurostat only provides data on police officer numbers.

(25)    CEPOL’s participant’s numbers are compared only to police officers, despite CEPOL target audience includes prosecutors and non-police investigators (e.g. lab personnel).

(26)    No data prior to 2016.

(27)    No disaggregated data on implemented activities available for 2018. Data in orange represents activities offered in CEPOL training catalogue for the year.

(28)    No disaggregated data on implemented activities available for 2019. Data in orange represents activities offered in CEPOL training catalogue for the year.

(29)    Only overall satisfaction for all training activities is available for 2016 and 2019.

(30)    Excluding from the total activities in third countries not covered by CEPOL’s dedicated budget.

(31)    The drop compared to the previous years is due to the cancellation of residential activities due to the COVID-19 outbreak and to the predominance of new COVID related activities offered by CEPOL (82 activities, 32% of total).

(32)    Activities in third countries have not been included in the total.

(33)    Participants to activities in third countries have not been included in the total.

(34)    Please see a more detailed illustration of the case studies at p. 58.

(35)    Please see a more detailed illustration of the case studies at p. 58.

(36)    Please see a more detailed illustration of the case studies at p. 58.

(37)    Webinars and online courses appear to be grouped under ‘e-learning activities’ in 2019 and 2020.

(38)    CEPOL, 2020, CEPOL Digitalisation Strategy Project “Future learning and development framework for CEPOL”.

(39)    Nine surveyed training participants explicitly mention LEEd as an improvement over the previous system, and a factor that enabled CEPOL to achieve its objectives.

(40)  No disaggregated data available. CEPOL’s CAAR 2020 aggregates Training activities in the area of Law enforcement leadership and management, Language skills and Train the trainers.

(41)  No disaggregated data available. CEPOL’s CAAR 2020 aggregates Training activities in the area of Law enforcement leadership and management, Language skills and Train the trainers.

(42)    Interviews with one Member State representative (#12) and one EU representative (#9).

(43)    Interviews with two EU representatives (#13, #16) and one JHA agency representative (#4).

(44)    Please see a more detailed illustration of the case studies at p. 58.

(45)    Joint Statement of the European Parliament, the Council of the EU and the European Commission on decentralized agencies (available here ).

(46)    Please see a more detailed illustration of the case studies at p. 58.

(47) The Single Annual report is called Consolidated Annual Activity report (CAAR). Art 9. Adds that the CAAR should be submitted also the national parliaments.

(48)    Please see a more detailed illustration of the case studies at p. 58.

(49)    Please see a more detailed illustration of the case studies at p. 58.

(50)    Calculation made drawing from data presented in CEPOL Consolidated Annual Activity Reports (2015, 2016, 2017, 2018, 2019, 2020).

(51) Interview with one EU-level representative (#16).

(52)    Please see a more detailed illustration of the case studies at p. 58.

(53)    Please see a more detailed illustration of the case studies at p. 58.

(54)    Please see a more detailed illustration of the case studies at p. 58.

(55)    Council Framework Decision 2002/584/JHA of 13 June 2002 on the European arrest warrant and the surrender procedures between Member States.

(56)    A rogatory letter is a formal request from a court in one country to ‘the appropriate judicial authorities’ in another country requesting compulsion of testimony, documentary or other evidence, or effect of service of process.

(57)    The European Agenda on Security 2015-2020 called for strengthening the links between EU Justice and Home Affairs and the Common Security and Defence Policy (CSDP), stressing that EU internal security and global security are mutually dependent and interlinked. One of the ways this link has been progressively reinforced is through the launch of Common Security and Defence Policy missions which, since the first European Union Police Mission was launched in Bosnia and Herzegovina in January 2003 have deployed EU civilian and military personnel abroad with the aim of promoting stability and contain potential and effective threats to EU internal security (European External Action Service website: https://eeas.europa.eu/archives/csdp/missions-and-operations/eupm-bih/index_en.htm ).

(58)    Please see a more detailed illustration of the case studies at p. 58.

(59)    Q19: 3 = strongly improved prioritisation of CEPOL’s training sessions, 5 = somewhat improved prioritisation of CEPOL’s training sessions, 1= not improved prioritisation of CEPOL’s training sessions , 5= Do not know (Survey of Framework partners, n = 14).

(60)    Regulation (EU) 2015/2219.

(61) Elements listed in the table were reported only by less than 25 stakeholders (both CNUs/NCPs and participants), while the overarching majority of survey respondents from these categories of stakeholders either did not have anything to report or were completely satisfied with the CEPOL’s training offer. Nothing was reported by CEPOL Framework Partners.

(62)    Please see a more detailed illustration of the case studies at p. 58.

(63)    Number of unique users answering the survey.

(64)    As the survey was circulated by CNUs among participants to CEPOL training activities, it is not known how many participants have been targeted. The estimated number of possible targets is approximately 2.5 million.

(65)    Responses from both the CNU and the NCP were collected from AT, DE, ES, HU, MT, RO, SK. For the other (19) countries, either the CNU or the NCP provided the response.

(66)    CZ, DE, FR, HR, LT(2), LV, PL, PT(2).

(67)    Armenia, Lebanon (2), Ukraine.

(68)    AT(31), BE(28), BG(5), EE(7), ES(22), FI(21), GR(15), HR(21), HU(18), LV(5), LU(30), NL(18), PL(6), PT(1), SI(16), SK(10), SE(4).

(69)    The number drops to 44, considering only CNUs/NCPs, Framework partners and Third countries.

(70)

OJ L 319, 4.12.2015.p.1

(71)

1]  Regulation (EU) 2015/2219 of the EP and of the Council on the European Union Agency for Law Enforcement Training (CEPOL)