EUROPEAN COMMISSION

Brussels, 2.6.2023

SWD(2023) 201 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

of the



EUROPEAN CRIME PREVENTION NETWORK



2012-2022

{SWD(2023) 202 final}

Table of contents

1.Introduction

2.What was the expected outcome of the intervention?

2.1    Description of the intervention and its objectives    

2.1.1. Policy Context    

2.1.2. The EUCPN: goals, tasks and structure    

2.1.3. Intervention Logic    

2.2 Point of comparison    

3.How has the situation evolved over the evaluation period?

3.1.    Facilitate cooperation, contacts and exchanges of best practices    

3.2.    Develop knowledge and expertise on crime prevention    

3.3.    Disseminate knowledge and expertise to facilitate implementation    

3.4.    Contribute to various aspects of crime prevention at the EU level    

4.evaluation findings

4.1.    To what extent was the intervention successful and why?    

4.1.1.    Effectiveness    

4.1.2.    Efficiency    

4.1.3.    Coherence    

4.2.    How did the EU intervention make a difference?    

4.3.    Is the intervention still relevant?    

5.What are the conclusions and lessons learned?

5.1.    Conclusions    

5.2.    Lessons learned and suggested areas for improvement    

Annex I: Procedural Information

1.    Lead DG, Decide Planning/CWP references    

2.    Organisation and timing    

3.    Exceptions to the better regulation guidelines    

4.    Evidence and sources    

Annex II. Methodology and Analytical models used

1.    Methodology    

2.    Timing    

3.    Limitations    

4.    Commission assessment of the work of the contractor    

Annex III. Evaluation matrix

Annex IV. Overview of benefits and costs

Annex V. Stakeholders consultation - Synopsis report

1.    Objectives of the consultation    

2.    Consultation methods and tools    

3.    Results of the consultation activities    

4.    Interviews and attendance in workshops    

Effectiveness    

Efficiency    

Coherence    

EU added value    

Relevance    

5.    Stakeholder survey    

Effectiveness    

Efficiency    

Coherence    

EU added value    

Relevance    

5.    Public Consultation    

Glossary

1.     Introduction

Organised crime is a real and increasing threat to our societies, democracies and economies. As organised criminal groups look to sustain and expand their activities, they take advantage of their position to recruit their members and leaders of tomorrow. Individuals raised in criminal environments and in socio-economically deprived areas are most vulnerable to recruitment for criminal activities. Most activities within the criminal justice system, such as investigations, prosecutions, and the imposition of sanctions are carried out after the crime has been committed. However, in order to prevent people from joining a life of violence and crime, and in order to raise the resilience of society, it is paramount to invest in crime prevention policies and tools. 

A multidisciplinary and inter-agency approach to crime prevention can contribute to reducing the likelihood of criminal activity, by raising awareness among potential victims, or providing support to vulnerable individuals and communities to reduce the risk of falling into a criminal lifestyle. Crime prevention involves a wide range of stakeholders, including schools, families, the community, the social welfare sector, civil society and the private sector; and their cooperation and partnerships are of vital importance. Therefore, as laid down in the Organised Crime Strategy 1 , the Commission aims to enhance the exchange of knowledge and best practices on crime prevention through the European Crime Prevention Network (hereafter referred to as EUCPN or the Network).

The EUCPN was set up in 2001 by the Council of the European Union. 2  Currently, Council Decision 2009/902/JHA 3  (‘the Council Decision’) sets out the aim to develop a platform for the Member States to exchange knowledge and best practices in the field of crime prevention, with the aim to reduce or otherwise contribute to reducing crime and citizens' feeling of insecurity 4 . The Network supports tangible crime prevention activities and contributes to formulating EU policy and strategy on crime prevention. The EUCPN was set up to be a point of reference for its target groups: practitioners 5 and policymakers working on crime prevention at local and national level, as well as for policymakers at the EU and international level. The Network is funded by the European Commission and contributions from the Member States.

10 years since its last evaluation in 2012 6 , the EU and national policies to counter crime have undergone significant developments. Crime prevention has been fully recognized at the EU level to be an essential component of an effective strategy to tackle crime. The preventive dimension of crime has been included in several EU policies, from drugs, to trafficking in human beings, radicalisation and child sexual abuse. Furthermore, the network underwent important changes: EUCPN has aligned its crime prevention activities with the priorities of the European Multidisciplinary Platform against Criminal Threats (EMPACT) 7 , the EU flagship instrument for multidisciplinary and multiagency operational cooperation to fight organised crime at EU level. In order to ensure the adequate involvement of the Network in the EMPACT platform, the Network’s Secretariat was strengthened from 2 to 7 full-time members, and the Network’s budget was increased up to EUR 2 million for a period covering two years. 8  

In light of these and more developments, it was decided to perform a new evaluation and an assessment of the implementation of recommendations in 2012, as recommended by the 2012 evaluation, to ensure that the available knowledge and practical tools on crime prevention are used to their full potential.

This staff working document (SWD) presents the findings of the evaluation of EUCPN for the period 2012 - 2022. In line with the Better Regulation guidelines, the Commission assesses the effectiveness, efficiency, coherence, EU added value and relevance of the Network. The evaluation covers the Network’s achievements since the last evaluation in 2012, and provide the useful evidence base to address weaknesses.

In order to provide the evidence base for the evaluation, a study of the Network’s progress and results since 2012 was conducted by an external contractor between March 2019 and June 2021 (‘the external study’). This study involves all Member States 9 . A wide range of stakeholders was consulted as part of the study. These included Member States competent authorities at the national, regional and local levels; the Network’s members, crime prevention practitioners and policy makers, academia and researchers, as well as relevant Commission services, and EU agencies.

In order to carry out the external study, a range of methodological tools and techniques were used. These included the review of 164 documents, 58 interviews with relevant stakeholders (EUCPN members, law enforcement, local authorities, academics, NGOs, EU institutions and agencies), a dedicated online survey targeting the same stakeholders, 4 specific case studies on selected actions of the EUCPN Multi-Annual Strategy 2016-2020, an analysis of 12 comparable EU networks 10 , and a public consultation through the Have Your Say website of the Commission.

Several limitations were identified throughout the course of the evaluation. Not all information and data on aspects relevant to the evaluation was available, especially about costs relevant for the assessment of efficiency, such as the hours/days spent on goals and actions of the Multi-Annual Strategy. Moreover, the sources identified at national level, such as national laws and institutional websites, provided a rather heterogeneous pool of information, which made it difficult to make clear comparisons. There were also several limitations on the information collected with the field research. One of the main issues was the high turnover among National Representatives, making it more difficult to acquire information from experienced National Representatives. These limitations were overcome with specific mitigation measures, for example by interviewing former Representatives and the former head of the Secretariat 11 .

This evaluation covers the period from the EUCPN’s last evaluation in 2012 12 until September 2022. Developments after the completion date of the external study have been included in this evaluation by the Commission. The descriptions of the most recent developments are based on fact finding, based on open sources and interviews and contacts with the EUCPN Secretariat and the National Representatives.

2.    What was the expected outcome of the intervention?

The European Crime Prevention Network was set up primarily in order to exchange and develop knowledge and best practices on crime prevention at the local, national and EU level. This section describes the policy context, how the Network is structured and works towards achieving its objectives, and the intervention logic. The section concludes with the 2012 evaluation 13 findings as the point of comparison.

2.1     Description of the intervention and its objectives

2.1.1. Policy Context

The EU’s policy on crime prevention focuses on facilitating exchanges of experience and best practices to mitigate factors that encourage crime and recidivism, including on preventing people from ending up in vulnerable situations that may lead them to engage in criminal activity. The Security Union Strategy 14 names close cooperation with local and regional administrations as well as civil society as key for crime prevention. Furthermore, the EU works towards prevention of corruption as well as of criminal infiltration of the economy and society Prevention is an integral part of EU policy on cybercrime 15 , anti-radicalisation 16 , environmental crime 17 , drugs 18 , trafficking in human beings 19 , child sexual abuse 20 , domestic violence 21 and firearms. 22  

Since crime prevention includes all the activities that contribute to halting or reducing crime as a social phenomenon, its definition can be broad, and may depend on the specific context. In 2019, the Network defined the concept of crime prevention as ‘Ethically acceptable and evidence-based activities aimed at reducing the risk of crime occurring and its harmful consequences with the ultimate goal of working towards the improvement of the quality of life and safety of individuals, groups and communities’. The Network classifies crime prevention into four different types of interventions 23 . In developmental prevention, the goal is to prevent the onset of criminal behaviour. Insights from psychology, education, medicine and public health can guide these activities, for example with parenting and early childhood support. Secondly, community prevention aims to influence behaviour by changing the social conditions and institutions in the community. Examples of community prevention are neighbourhood watch programmes and police outreach to the public. Thirdly, situational prevention has a prime interest in the criminal event itself. By changing conditions, such as installing alarms, the opportunity to commit a crime and/or the likelihood to be victimized is reduced. Lastly, criminal justice interventions mostly take place after crime has been committed, but can also have a future-oriented dimension, for example with general deterrence and rehabilitation programmes.

In the Organised Crime Strategy, it is stated that the Commission will enhance the exchange of knowledge and best practices on crime prevention through the EUCPN. The Network’s activities and outputs involves crime prevention in all its aspects, as described above. The next section explains how his is done, by elaborating on the Network’s mandate, objectives, tasks and structure.

2.1.2. The EUCPN: goals, tasks and structure

The overall objective of the EUCPN, set by Council Decision 2009/902/JHA, is to contribute to ‘developing the various aspects of crime prevention at the Union level, in accordance with EU policy on crime prevention’. The Network shall support crime prevention activities at the national and local level, as laid down in Article 2.1 of the Council Decision. It will cover all measures that intend to reduce or contribute to reducing crime and increasing citizens’ safety, while acknowledging the multi-disciplinary nature of crime.

In the pursuit of the strategic objectives of the EUCPN, the Network is headed by a Board of National Representatives from each EU Member State, chaired by the rotating Presidency of the Council of the European Union. The National Representatives are civil servants from national administrations, who carry out this task next to their nationally appointed position. The Board is supported in its activities by an Executive Committee 24  and the Secretariat. The Secretariat aims to provide general administrative, technical and analytical support to the Network, and represents the Network externally. The European Commission acts as observer, both in the Board and in the Executive Committee. Since 2011, the Network is supported by an enlarged Secretariat whose functions and role have been progressively strengthened over the years. The Secretariat now consists of 7 full-time members.

In order to implement its objectives and tasks, the EUCPN adopts five-year Multi-Annual Strategies (MAS) 25 . Concrete objectives and tasks are laid down in Annual Work Programmes 26 . The Secretariat drafts both the MAS and the Annual Work Programmes, with input from the National Representatives, after which they are formally adopted by the Board. The Secretariat monitors progress and results with feedback from the National Representatives and writes yearly reports of its annual programmes.

The Commission has provided financial support through financial instruments in the area of justice and home affairs, including the Prevention of and Fight against Crime (ISEC) 2007-2013 27 programme and the Internal Security Fund 2014-2020. Under the new Multi-Annual Financial Framework (MFF) 2021-2027, the Commission is supporting the Network with 24-month direct grants of approximately EUR two million under the Internal Security Fund Police. 28 Member States provide structural financial support through the co-financing mechanism (5% of the total amount, divided according to GDP) under this EU funding programme and can fund specific projects of interest to the Network. In the table below, an overview of the amounts given to the EUCPN is provided:

2.1.3. Intervention Logic

The intervention logic provides an overview of how the objectives of Council Decision 2009/902/JHA were to be achieved through the work of the EUCPN. The logic used for this evaluation is summed up in the figure below, showing the needs addressed.

2.2 Point of comparison

The baseline for the EUCPN was mainly constructed by relying on the previous evaluation of the Network conducted by the Commission in 2012 29 , with the support of an external study 30 . This provides an adequate frame of reference, as the objectives set by for the EUCPN by the Council of the European Union have not changed. The 2012 evaluation stated that following the adoption of the 2009 Council Decision and follow-up activities, the Network had undertaken successful steps to foster better cooperation, contacts, and exchange of information, and the quality and quantity of its outputs had improved.

However, the evaluation also identified a number of weaknesses:

(1)insufficient evidence that EUCPN activities are always linked to the crime prevention priorities facing the EU and Member States;

(2)EUCPN is in general less successful in reaching target groups at local level than at national or EU level;

(3)a substantial amount of crime prevention projects supported by the ISEC programme are implemented without associating or even informing EUCPN;

(4)the intention to put key documents in different EU languages on the website to attract a wider readership and provide supporting documentation to stakeholders particularly at local level was not fulfilled;

(5)many of the contact points do not sufficiently contribute to the functioning and visibility of the EUCPN and in a variety of Member States there are no contact points;

(6)more needs to be done to raise the EUCPN profile in general.

The objectives set out in the Council Decision and the 2012 recommendations have set out clear expectations for the development of the Network. The EUCPN should:

-further facilitate the cooperation and exchange of practices between all relevant stakeholders;

-establish closer relationships with relevant partners;

-take a more strategic approach towards contributing to EU, national and local crime prevention needs, while improving the evidence base of its products;

-work to strengthen its contribution to implementing effective crime prevention measures and policies at national, regional and local level;

-monitor and support the impact of crime prevention measures;

-ensure a broad dissemination and awareness of its products and activities;

-further align its priorities to agreed EU priorities and strategies and contribute to the development of EU policies and projects in the field of crime prevention.

In terms of the organisational set up, set out in the Council Decision, the Secretariat was expected to play a key role in supporting the activities of the Network, while the National Representatives were expected take a more active role in developing its activities.

3.    How has the situation evolved over the evaluation period?

This chapter describes the development of the EUCPN since the 2012 evaluation, and how the Network has worked towards achieving its intended goals, as set out in the intervention logic.

3.1.    Facilitate cooperation, contacts and exchanges of best practices 

The EUCPN aims to provide a platform for practitioners and policy makers to connect and exchange best practices across the EU, by organizing conferences, seminars and by providing support to national, regional and local stakeholders. In order for these services to reach the intended audience, the EUCPN has taken a number of measures in order to facilitate exchanges between practitioners at the national and local level.    In order to facilitate the exchange of best practices, the EUCPN organizes several events. The EUCPN organizes an annual conference for sharing and disseminating experience and knowledge of Best Practices in preventing crime and increasing safety and security in the EU Member States (BPC). Here, the European Crime Prevention Award (ECPA) is handed out, for which Member States can propose crime prevention projects on a theme chosen by the Presidency of the Network. Furthermore, the EUCPN organizes biannual Crime Prevention Conferences, where all the target groups of the Network (policy makers and practitioners at EU, national and local level) meet to exchange knowledge on a chosen topic.    Aside from events and conferences, the Secretariat also reaches out to and connects with practitioners directly. 31 The Secretariat has participated in many international, regional and local events to present its products. Moreover, the Secretariat was involved in other organisations' events and activities, such as the European Network for the Administrative Approach (ENAA), CEPOL, Europol and Eurostat. In order to improve cooperation with other networks and organisations, the Secretariat has engaged with several other entities by organising dedicated stakeholder meetings, or by attending relevant events. 32  

According to the division of labour laid down in the Council Decision, each National Representative shall promote the Network’s activities at the national and local level and can be supported by National Contact Points in doing so. However, National Representatives have not been able to take up a proactive role in creating a network of local and national contact points and their input to the work of the EUCPN has been relatively modest. According to the findings, most National Representatives face difficulties in reaching out to stakeholders due to the lack of translated material, and their limited capacity for outreach due to the requirements of their position in the national context.

   However, the Secretariat has created occasions for the development of a network of local and national contact points, such as the meeting between the National Crime Prevention Councils, which was first organised in March 2018. Moreover, the Secretariat regularly participates in and shares expert groups and workshops, for which experts from Member States are invited 33 .

3.2.    Develop knowledge and expertise on crime prevention

The Network has strived for a common approach to research on crime prevention, by establishing an agreed definition of crime prevention in 2019, in consultation with practitioners, academics, and stakeholders: ‘Ethically acceptable and evidence-based activities aimed at reducing the risk of crime occurring and its harmful consequences with the ultimate goal of working towards the improvement of the quality of life and safety of individuals, groups and communities.’ This development is part of a stronger shift and emphasis on evidence-based policy making. 

In order to contribute to the development and provision of knowledge and expertise to policy makers and practitioners, the Network produces a variety of written outputs, which are addressed to different target audiences: monitors, policy papers, toolboxes, reports and information about funding sources. 34  The EUCPN publishes these outputs on its website. 35  

The range of activities and topics is wide and can encompass all the dimensions of crime prevention policy mentioned in section 2.1.1. The Toolbox on street gang prevention (2022) 36 for example, covers developmental prevention (family-oriented interventions, among others), community prevention (social development in neighbourhoods) and criminal justice prevention (disengagement and rehabilitation). There are many more topics, ranging from cybercrime, to trafficking in human beings, to community-oriented policing, for example. The EUCPN publishes these outputs on its website. 37  

   EUCPN products are varied but categorised according to their contents and intended use. First, the EUCPN Secretariat, in cooperation with the EUCPN Presidencies, conducts research and analysis on the themes selected by the Presidencies. The outputs stemming from these activities include Toolboxes, which are published twice a year and include practical tools for practitioners for addressing the Presidencies’ themes, and thematic papers that provide information for policymakers for helping them deepen their knowledge on the selected themes.

Secondly, the EUCPN carries out research activities independent from the themes selected by the Presidencies, whose results are published in booklets, research reports, and monitors that present recent data and preventive measures for facing crime phenomena. In order to support evidence-based policy making, the Secretariat has published ‘Mythbusters’ that aim to dispel common misconceptions about crime prevention.

Furthermore, the Network produces material for conducting crime prevention campaigns at the European, national, and local level. Finally, the Secretariat collects information, on national crime prevention policies and strategies and on EU-funded projects and, mainly, on procedures to apply for the funding. However, information about national crime prevention strategies is incomplete and sometimes outdated, and the information on interregional information exchange, for which the Secretariat started working on a paper to illustrate policy similarities between states and regions in the EU, is limited. 

In 2022, the EUCPN has made further steps to improve the evidence base of their products. The Network announced the establishment of an Advisory Board, consisting of academic experts, who can provide independent scrutiny and expert peer review on the commissioning of research. 38  Furthermore, the EUCPN has founded a dedicated working group, which 39 developed a minimum list of criteria for crime prevention which will have to be met it can be disseminated by the EUCPN as a ‘good practice’. 40

3.3.    Disseminate knowledge and expertise to facilitate implementation

The Network strives to play an important role in the dissemination of qualitative knowledge on crime prevention, which could be shared to enable to improve crime prevention policy and tools in the EU. This entails collecting, sharing and communicating information on relevant crime-related problems, interventions and good practices, as well as on effective means of implementation.

Regarding outreach and communication, the Network established a communication plan in 2018, which has resulted in the creation of a new website, an increased presence on social media (Twitter, LinkedIn, Facebook), an updated newsletter, which has been disseminated more and more (in 2018 6 newsletters, in 2019 13 newsletters and in 2020 15 newsletters), the production of promotional material and the creation of videos, which can be viewed on the website. The website is the main platform for knowledge dissemination and contains a ‘knowledge centre’ database where users can find EUCPN outputs such as research articles, toolboxes and campaigns.

Regarding dissemination, the National Representatives have reported to face difficulties in fulfilling their role, according to interviews conducted by the external study: three National Representatives declared they have no channel to directly communicate with local stakeholders and are unaware of whether the information they share at the national level ultimately reaches the local level. Among surveyed stakeholders, 41% of responding national, regional or local authorities declare to not receive information from their National Representatives. For local policymakers and practitioners, this share increases to 80%. 41 The interviewed stakeholders report no instances of feedback requests from their Representatives or the EUCPN in any capacity.

In order to partially mitigate the challenges in reaching out to the national and local level, the EUCPN Secretariat has increased its presence on social media in order to reach the local practitioners without intermediaries. However the language barrier remains an issue as the EUCPN communicates and produces output almost entirely in English.

Translation of materials is a key factor that affects the dissemination of materials. The range of outputs of the EUCPN suffers from a scarce availability of material translated into national languages. The Network is aware of the importance of providing translations. However, up to now, only few documents are translated in all EU languages, due to a lack of resources. 42  Even though on occasions, National Representatives have facilitated translations, they reported that usually there is no budget earmarked for this. Since the finalisation of the external evaluation, the Secretariat has continued their work on knowledge dissemination, by organising webinars and trainings. Furthermore, the EUCPN has worked on developing trainings for practitioners. In 2022, the EUCPN Secretariat will develop a training on crime prevention knowledge and evidence, for which National Representatives are responsible for the further dissemination to their Member State. 43    

Regarding implementation, the 2012 evaluation recommended that the EUCPN should strengthen its role in making inputs to policy making in the crime prevention field. However, the EUCPN does not systematically monitor whether Member States adapt best practices implemented in other Member States. Moreover, interviews from the external study review that stakeholders lament the lack of support to adapt best practices to the local context. The Network acknowledges its lack of capacity in this regard.

3.4.    Contribute to various aspects of crime prevention at the EU level

From a strategic perspective, in 2015 the Network decided to align its activities to the priorities of the European Multidisciplinary Platform against Criminal Threats (EMPACT). This decision enabled it to put the emphasis on the illicit activities carried out by organised crime groups. Since 2018, the EUCPN has participated in operational actions carried out within the EMPACT priorities organised property crime, trafficking in human beings, child sexual exploitation and environmental crime.

In the new EMPACT cycle 2022-2025, approved by the Member States, the EUCPN is tasked to coordinate the Common Horizontal Strategic Goal 7: Prevention and harm reduction, assistance to victims, awareness raising. The Secretariat represents the Network in EMPACT meetings and can advise and support the drivers and action leaders on preventive actions. Furthermore, the Secretariat can propose preventive actions for priorities in which there is no preventive action yet. Moreover, depending on the resources available, the Secretariat could also take the lead in EMPACT priorities.

   In terms of EU level cooperation, the EUCPN mainly collaborates with the European Union’s Law Enforcement Agency (Europol), notably in the framework of EMPACT, and the European Union Agency for Law Enforcement Training (CEPOL). On 23 June 2020, a Working Agreement with CEPOL was signed. The Agreement allows participation of the EUCPN Secretariat in CEPOL training sessions and CEPOL staff in EUCPN activities and enhances the dissemination of crime prevention initiatives and research. The 2012 evaluation recommended a closer cooperation with Eurostat in order to gather statistical information on crime prevention at the EU level. The EUCPN continued to monitor its cooperation with Eurostat until 2017, but has not continued this cooperation on a systemic basis.

In the Council, the EUCPN regularly presents its activities. In 2018, the EUCPN addressed the Standing Committee on Operational Cooperation on Internal Security (COSI) in relation to Community Oriented Policing. A position paper was drafted together with CEPOL to explain the training plans. The EUCPN regularly presents its activities to the Law Enforcement Working Party (LEWP).

4.    evaluation findings 

4.1.    To what extent was the intervention successful and why? 

4.1.1.    Effectiveness

The section below analyses whether the four intended results as set out in the intervention logic have been obtained.

Cooperation, contacts and exchange of best practices

Since 2012, the EUCPN created new and more varied opportunities to facilitate cooperation, contacts and exchanges between practitioners and policymakers at EU, national and local level. The National Representatives use their contacts in order to reach out to practitioners, and some events, such as the European Crime Prevention Conference, are advertised on the EUCPN website and social media. 44 Furthermore, the Secretariat participates in a range of international, regional and local events in order to reach practitioners directly and present its products. The Network has strengthened its cooperation with several organisations working on crime prevention, notably by organising dedicated stakeholder meetings.

   The different tools the EUCPN uses are valued differently and in a different way across categories of stakeholders: some products are seen as more useful than others. The capacity of the Network to organise meeting opportunities stands out as favourite output among stakeholders. 45  Interviewed target groups praise the network for sharing practical toolkits, thematic insights and best practices, and for providing networking and knowledge-sharing opportunities.

   However, there are some differences in the perceived benefits per stakeholder group. National Representatives and national authorities are generally satisfied with the opportunities provided for cooperation and exchange of information, while local stakeholders find the services of the EUCPN comparatively less relevant and useful. Generally, the Network seems to have difficulty in reaching target groups below national level. Among surveyed stakeholders, 41% of responding national, regional or local authorities declare to not receive any information by their National Representatives. When looking at local policymakers and practitioners, the share jumps to 80%, which aligns with the increased dissatisfaction with the ability of National Representatives to interact with the local level compared to the past, when practitioners and academics were more involved in the activities of the Network.

The development of knowledge and expertise on crime prevention

Based on the positive responses received from stakeholders, the EUCPN effectively collects and produces qualitative knowledge on crime prevention and has managed to raise the quality of outputs produced through the continuous collection and dissemination of good practices. Stakeholders also report indirect benefits resulting from the work of the EUCPN. They report using the Network as a platform to scout for partners for EU funded projects, they interpret EUCPN materials in order to know what the key topics of interest are when writing project proposals, and they use evaluated best practices shared by the Network for prevention activities within their own administration.

Concerning the EUCPN’s efforts to develop a common approach to crime prevention, the 2019 adopted definition of crime prevention resonates with and aligns to stakeholders’ understanding of crime prevention. The vast majority of surveyed stakeholders (75%) recognised the contribution the EUCPN is making towards spreading the culture of evaluating crime prevention activities.

   The external study examined whether policymakers and practitioners find the materials useful. The publication of national crime prevention strategies on the EUCPN website are considered to be quite useful by stakeholders, but reports are considered of less practical use. Respondents have a clear preference for practical toolboxes. The target group that finds EUCPN materials to be the most useful are the national authorities, while the target group that finds them the least useful are local practitioners.

Knowledge dissemination and implementation of crime prevention policies and tools

The new EUCPN communication strategy has been effective in raising awareness about the Network, both at EU and national level, as concluded following interviews carried out in the external study. The website is the main gateway to promote the results of the Network and represents a drastic improvement both in terms of accessibility and production value over the one operating in 2012. Through its social media presence, the EUCPN Secretariat is able to reach out to its target groups directly. This is corroborated by survey responses, as 21 respondents out of 72 (30%) found social media channels as being “highly” or “very highly” effective in disseminating information on the Network, with an additional 29 (40%) considering them moderately effective. Only 22 respondents considered the effectiveness of social media channels as having a “low” or “very low effectiveness”. Amongst the latter, about half (10) of responses came from academics. The website was also positively assessed by survey respondents, with 46 out of 76 respondents (60%) considering it as being “highly” or “very highly” effective in disseminating information on the Network.

However, these developments are not enough to reach all target groups. According to the outcomes of stakeholder interviews, problems with dissemination are attributed to a combination of factors, some linked to the Network itself, such as the limited capacity for active involvement of the National Representatives in general, and the obstacles they face to efficiently disseminate materials and knowledge to the local level. Moreover, the Network’s limited resources to translate materials in national languages is acknowledged by stakeholders as an important factor hindering the ability of the EUCPN to disseminate its products and is the most commonly reported impediment.

Another factor hindering the Network’s outreach is the lack of feedback collection from their target groups in order to produce output tailored to their needs, such as support to contextualising best practices, guidance on evaluating crime prevention projects, or simply translated materials. This explains in part why local stakeholders make the least use of EUCPN knowledge products and consider the Network to be adding the least value compared to national and local initiatives. 

Here again, the EUCPN is not sufficiently equipped to reach its intended user base. Target groups that are more easily reached by the EUCPN are also the most satisfied by its activities. Stakeholders would like the Network to provide guidance on how to replicate best practices in different contexts, and how to approach application at a local level.

Some of the obstacles hindering the EUCPN ability to contact some of its target groups are external to the Network: they can be of cultural, administrative or organisational nature, and little can be done by National Representatives to overcome them. Not all Member States can count on National Crime Prevention Councils to facilitate the dissemination of EUCPN materials. Another possible obstacle is the fragmentation in administrative structures: the more decentralised governments operate, the more difficult it is to reach the local level. When comparing the EUCPN to other networks, as done in the external study, the EUCPN is disadvantaged compared to other networks due to its weak dissemination channels. Other networks can count on more established national and local relays. 46  

Finally, crime prevention as a topic can be of varying importance per Member State: if there is no strong prevention culture present, or if crime prevention is not as high on the political agenda as other security topics, this will have an impact on the amount of attention devoted to the EUCPN. 47  

Contribution to EU crime prevention policies and actions

The EUCPN has been working on building the foundations to become a stronger point of reference on crime prevention in the EU, by aligning its priorities to EU priorities, by taking part in EMPACT, and developing working relationships with EU agencies. Since EUCPN only started as coordinator of the preventive angle for the 2022-2025 Cycle, it is too early to see the results of its efforts as coordinating entity.

   The Network has followed a structured approach to reach out and establish relationships with other EU actors in the field of security, and now mostly cooperates with EUROPOL in the framework of EMPACT and joins efforts with CEPOL in providing information and trainings on crime prevention.

   Still, EU stakeholders indicate that the Network needs to improve on its outreach and expertise in order to successfully contribute to EU policy making on crime prevention. The Network’s contribution is generally appreciated by EMPACT stakeholders, 48  but interviewees noted that the EUCPN does not always bring the required expertise to the table, therefore limiting the impact of its contribution. Based on interviews carried out in the external study, it was noted that National Representatives rarely offer inputs for the EUCPN in EMPACT.    

   Even so, there is untapped potential for the Network to contribute further to EU crime prevention policies and actions, in terms of coordinating input from its members and intensifying contacts with relevant stakeholders. Furthermore, there is scope for further efforts to increase synergies with EU policies and strategies. In stakeholder interviews, involved parties appreciate the Network for bringing the prevention perspective to the table and developing operative actions and quality outputs promoting EMPACT priorities. 

4.1.2.    Efficiency

In order to establish whether the Network is efficient, the section below analyses the European Crime Prevention Network’s cost-effectiveness in executing its tasks, as well as the appropriateness of the allocation of its resources to implement those tasks.

   The assessment of the efficiency of the Network was influenced by several limitations. Both the subject matter of crime prevention (how to measure whether a crime has been prevented?) and the limitations to the amount of data the external study was able to collect, made it extremely difficult to quantify the EUCPN’s cost-effectiveness. Nevertheless, an attempt was made to provide a qualitative overview based on stakeholders’ views.

The main question to gauge the point of view of stakeholders concerning the overarching efficiency of the Network was the following: “to what extent costs and benefits of the Network are proportionate?”. In total, 29 responses were received from EUCPN National Representatives or substitutes and EUCPN Secretariat members. A majority of EUCPN practitioners (19/29) which responded to the question reported that the costs and benefits of the Network are commensurate to a high or very high extent. In general, target groups of the Network bear virtually no cost, which explains why the Network is unanimously considered good value for money.

For the National Representatives and Contact Points, the picture is different: they can face costs unaccounted for by the EUCPN budget, such as overtime work and financial costs for dissemination activities. Moreover, several National Representatives indicate that their role is an additional task on top of other full-time commitments. There are also Representatives who report that the administration they work for does not allocate budget for EUCPN activities, such as for translating and printing campaign materials. The effort and time spent by National Representatives to find the necessary financial resources is reported as a cost in itself. 

Moreover, the Network struggles with operational inefficiencies stemming from its organisational setup. The EUCPN is an Internal Security Fund - Police funded project managed by the Belgian Ministry of Interior, so the Secretariat has to navigate both the financial regulations of Belgium and the European Union. Especially in the past, this has led to large parts of the budget remaining unspent (see p. 10 for the overview). The 5% contribution from Member States to the Network, is also subject to issues. Member States’ delayed payments cause additional inefficiency.     

The implementation of EUCPN activities also encounters some inefficiencies, related to the division of roles and responsibilities within the Network. Even though the Council Decision and Rules of Procedure are clear in setting roles and responsibilities, the Secretariat takes on a disproportionate part of the responsibilities, in an effort to compensate for National Representatives’ limited involvement.

Notably, the implementation of actions under the Multi-Annual Strategy 2016-2020 fell almost entirely on the Secretariat. National Representatives appear to be only marginally involved or not involved at all, despite the fact that there are several actions that identify them as responsible for the implementation. The establishment of a permanent Secretariat in late 2011 and its further consolidation throughout the years might have initiated or compounded this behaviour.

   This causes inefficiencies, not only because the Secretariat is using its time and resources on tasks for which the other EUCPN members are better placed, but also because the National Representatives and the National Contact Points are the key to the EUCPN’s network function. Without their active involvement and contacts, the EUCPN’s is not sufficiently able to reach its target groups. Furthermore, members of governance share the opinion that the high turnover among National Representatives entails considerable costs for the Network and influences its ability to pursue its objectives, since new Members need time to familiarise themselves with the work of the EUCPN, and the Network is less able to draw from past experience.

4.1.3.    Coherence

The third component that contributes to establish the Network’s success is its coherence. Internally, this entails examining how the Network’s activities relate to one another and to its legal basis. Externally, the Network’s objectives in relation to other EU initiatives are evaluated, in order to identify possible synergies, inconsistencies and/or duplications.

Regarding the Network’s internal coherence, its objectives are sufficiently coherent with one another, and the actions carried out under the MAS 2016-2020 are generally consistent with and aligned to the objectives established by the Council Decision.

However, interviews with stakeholders show that in the Annual Work Programmes, there is a greater focus on achieving administrative tasks mentioned in the work programmes (such as setting up the website, updating the database of contacts etc.) than achieving the more overarching goals of the Network. In general, the belief within the Network has been that the Multi Annual Strategy 2016-2020 is too inward looking, as current actions are often aimed at strengthening the Network itself rather than providing policy orientation. Notably, both National Representatives and the Secretariat pointed out that the lack of cooperation between National Representatives influenced to some degree the ability to implement some of the actions of the Multi-Annual Strategy 2016-2020, indicating a lack of commitment towards achieving the Network’s common goals.

Regarding the Network’s external coherence, there have been many developments in EU policymaking. The Network’s intention to align itself more to EU policy has fostered more coherence with EU goals and priorities. From 2018, the preventive dimension became one of the Common Horizontal Strategic Goals (no. 7) of EMPACT, and in the EMPACT cycle 2022-2025, the Network became the coordinator of Goal 7. However, due to the Network’s recent involvement, the EUCPN needs more time to get fully integrated in EMPACT, and the Secretariat spends considerable time on trying to be included in the various EMPACT groups. Still, the Network’s participation in EMPACT has further potential to grow and develop, as the EUCPN gets more and more involved in its working groups and establishes contacts.     

   As the sole EU organization that exclusively focuses on crime prevention in all its aspects, the EUCPN is well positioned to complement the actions of EU Agencies active in the field of justice and security and works to build relationships with others. The relationship established with CEPOL and Europol, in particular, produced tangible results achieved in collaboration by the parties. However, the cooperation is not used to its full potential yet. According to interviews with the EU agencies, this may be either attributed to the early stage of the collaboration or to an uneven level of commitment existing between the parties. Moreover, it was pointed out that the EUCPN may risk overlapping or duplicating the activities of other EU activities when setting out its priorities. 

4.2.    How did the EU intervention make a difference?

The evaluation found that the EUCPN has generated EU added value as far as it achieved results that national or other EU initiatives would not otherwise have achieved:

·The EUCPN provided uniform access to the same services and knowledge products on crime prevention to its target groups;

·The Network facilitated exchanges between practitioners in the field, which would have not have otherwise occurred with the same regularity and to the same extent without EU intervention; When the EUCPN got involved in EMPACT, national and EU authorities could benefit from its readily available contributions to coordinated operational actions. 

Looking at possible alternatives to the EUCPN, interviewees 49 report no other comparable forum for meeting and discussing crime prevention topics in a cross-cutting, multi-disciplinary way. The initiative with the most point in common with the EUCPN is the European Forum for Urban Security (EFUS), although EFUS is mainly addressed to municipalities.

The EU added value is not uniformly perceived across all target groups. Stakeholders at the local level see less value in the EUCPN, either because they are not aware of the full extent of EUCPN products and service or because they consider them responding to their needs to a lesser extent than what is available domestically. The same is true for stakeholders from Member States with a strong culture of crime prevention, whose ‘domestic market’ already satisfies their needs.

In light of this, the most likely consequences of a reduction or withdrawal of EU involvement, with an interruption of the functions carried out by EUCPN, would entail greater difficulties for users to find and access the same, relevant information and services and less opportunities to exchange best practices among EU Member States. The impact of no EUCPN will fall disproportionally on national stakeholders that do not have a strong culture of crime prevention.

However, the potential for growth and development of the Network depends on the degree of political interest in Member States. Limited political interest can make it difficult for the Network to make crime prevention mainstream. Given the fact that the National Representatives strongly depend on the Secretariat, the cooperation and the coordination between Member States would likely not persist in the absence of the Network. Even so, as an EU network, the EUCPN should respond to the needs and requirements of the Member States, and if it does not do that sufficiently, its EU added value is at risk.

4.3.    Is the intervention still relevant?

Overall, the objectives of the Council Decision are still relevant to the EU needs of supporting the development of crime prevention measures in the EU. The framework devised by the Council Decision proved to be sufficiently flexible to enable the EUCPN to remain relevant over the years.

Compared to 2012, the EUCPN has increased its relevance at the EU level as it expanded the range of its activities to include emerging EU crime priorities in order to respond to the need to develop the various aspects of crime prevention at the EU level. All EU stakeholders recognise the EUCPN is making relevant contributions to EMPACT and works together with Europol and CEPOL in order to include crime prevention in EU activities.

   In order to stay relevant, the Network needs to respond to the needs of its target groups. However, the EUCPN does not systematically collect input or feedback from its target groups on their needs. Exceptions are the ‘EUCPN event feedback tool,’ which is limited to the participants’ opinions on the Best Practice Conference and the European Crime Prevention Conference. Similarly, there is no indication on activities or output produced by the Network which are tailored to the needs of national and local stakeholders.

Aside from aligning to strategic priorities at the EU level, there is no evidence that can indicate whether actions implemented have been relevant to the identified needs. EUCPN members have identified new challenges in board meetings, such as migration, 50 cybercrime, 51 big data 52 and radicalisation. 53  However, the topics chosen demonstrate that little attention is devoted to coherence with other EU initiatives.

For the national level, 70% of interviewees consider the activities of the Network to be relevant to the development of national strategies on crime prevention. 54 The respondents who answered positively indicated that the activities of EUCPN play a significant role in gathering knowledge concerning crime prevention, and that the principles and recommendation of international documents and best practices of EU Member States in the area of crime prevention are taken into consideration in the formulation of national crime prevention strategies. Stakeholders also mentioned that EUCPN activities such as conferences, meetings, seminars, campaigns and projects are extremely relevant in this field.

   On the local level, the EUCPN’s activities had close to no impact on local strategies and projects, also due to the limited awareness of the Network at the local level Generally, the group most satisfied with the work of the EUCPN is also the most represented within the governance, mostly consisting of representatives from national ministries and law enforcement agencies.

5.    What are the conclusions and lessons learned?

5.1.    Conclusions

Since 2012, the EUCPN has put in motion a long-term process that gradually consolidated its activities on several fronts, in particular by setting out Multi-Annual Strategies and Annual Work Programmes that help steer the different activities.

On the evaluation criteria, the following conclusions can be drawn:

·The EUCPN has been moderately effective in reaching its objectives and achieving the expected results. The EUCPN has been moderately effective in reaching its objectives and achieving the expected results in supporting the development of knowledge and exchange of best practices on crime prevention. The Secretariat has increased its efforts to facilitate cooperation and exchanges among practitioners and policymakers at the EU, national and local level. Furthermore, the Network has considerably developed its written output and increased its focus on research. Necessary steps have been made by the Secretariat to increase the Network’s visibility, with the renewal of the website and its active presence on social media (LinkedIn, Facebook and Twitter). The Network has aligned its activities with EU priorities, built relationships with relevant EU agencies such as Europol and CEPOL, and has worked on involving and embedding the preventive dimension within the EU, in particular through its involvement in EMPACT as coordinator of Common Horizontal Strategic Goal 7 on crime prevention.

However, there are several factors that prevent the Network from reaching its full potential. Firstly, outreach to its target groups, mainly to the local level, stays behind expectations, as the network and its output is not sufficiently visible, due to a lack of effective dissemination and tailoring to specific contexts. A second important factor hampering effectiveness is the lack of involvement and lack of structural feedback collection from the relevant stakeholders. Further efforts are also needed to increase synergies between the Network’s activities and other preventive initiatives at EU level.

·In spite of the limitations to data collection which influenced the study’s ability to assess the Network’s cost effectiveness, stakeholder surveys revealed that the EUCPN is an efficient initiative as it provides direct and indirect value to its target groups with relatively modest funding. However, the EUCPN faces operational inefficiencies, due to the limited resources available to enable National Representatives to fulfil their tasks. Due to its organisational structure, the EUCPN faces administrative obstacles when aiming to comply with both the EU and the Belgian internal financial procedures. Moreover, even though the distribution of tasks is clear, the active involvement of the National Representatives and Contact Points is limited, due to a lack of capacity and prioritisation, and the high turnover among National Representatives leads to a lack of continuity and experience to draw from.

·Regarding coherence, the Network is sufficiently internally coherent, in that its objectives and actions are aligned to the objectives of the Council Decision. Regarding coherence vis-à-vis other initiatives, the Network’s objectives are in line with the broader EU crime prevention policy objectives. However, both its involvement in EMPACT and cooperation with other actors are not used to its full potential yet. The development of prevention policies in the EU requires the EUCPN to increase its interaction with EU policymakers and to follow closely EU initiatives, set out in the different EU strategies and action plans as well as in EU funded projects.

·The EUCPN provides EU added value since it offers access to knowledge on crime prevention to policy makers and practitioners across the EU and facilitates exchanges of best practices between policymakers and practitioners from different EU Member States. However, EU added value differs per target group. Since the Network is less visible at the local level due to a lack of adequate dissemination, EU added value is lower for practitioners and policymakers at this level. Furthermore, not every Member State has the same culture of crime prevention, so EU added value is higher for Member States with less developed crime prevention policies.

·The objectives of the Council Decision are still relevant to address EU needs regarding the development and implementation of effective crime prevention policies and measures while providing enough flexibility to adapt to changing circumstances. Still, when deciding on priorities and relevant actions, the Network does not sufficiently take into account activities conducted by other EU networks, initiatives or projects playing a role in preventing criminality. This diminishes opportunities to complement and support other initiatives and in the worst case, leads to duplication of efforts. In addition, the Network does not systematically ask for input and collect feedback from its target groups, to ensure that its activities and products are relevant to all.

Overall, the EUCPN has progressed towards meeting its objectives, but stayed behind expectations in several areas. Systemic issues persist that affect the overall performance of the network, notably related to the involvement of the National Representatives, as well as regarding the EUCPN’s engagement with its intended audience, to ensure that its products are targeted and valued.

5.2.    Lessons learned and suggested areas for improvement

While the Network is expected to deliver in many strands of work and activities, given the limited resources and other obstacles identified in this evaluation, the Network needs to better structure its priorities and activities to increase the impact and visibility and generate effective results. A starting point should be an active involvement of the National Representatives as core elements of the Network as well as other forms of cooperation with relevant stakeholders at European, national and local level. The design of such activities should be part of the Multi-Annual Strategy, setting concrete actions as laid down in the Annual Work Programmes. Their implementation needs to be regularly monitored coupled with corrective measures where necessary.

Existing dissemination and communication channels have proven to be insufficient and need to be strengthened. This is where the National Representatives and Contact Points are of crucial importance. While their limited engagement has been an issue over many years and efforts have been undertaken to increase the level of engagement, the situation has still not significantly improved. Measures to address this could include:

-Commitments to implement and support actions as outlined in the Multi-Annual Strategy and Annual Work Programme, for instance through the appointment of National Representatives to lead specific actions to ensure leadership and accountability;

-Measures allowing National representatives to fulfil their role, including guidance by the EUCPN Secretariat on the intended target audience of EUCPN deliverables, Member States making available the necessary resources and the development of an introduction programme for National Representatives and Contact Points.

Facilitating cooperation and exchanges between relevant stakeholders, providing evidence-based knowledge and guidance that help the stakeholders designing and implementing the necessary crime prevention measures crucially relies on a proper assessment of the needs, expectations as well as feedback channels on the usefulness and added value of EUCPN deliverables. It is therefore suggested that the EUCPN:

-Routinely maps Member States’ priorities and needs, with a structured approach that could foresee the inclusion of up-to date information regarding national crime prevention strategies to be available for consultation, on the EUCPN website;

-Provides for opportunities for feedback from National Representatives and Contact Points on their use of EUPCN materials and services, to ensure that the Network meets the needs of its members;

-Collects and processes relevant feedback from target groups to develop better-tailored products and services.

The effective implementation of crime prevention measures requires the development of tools that help adapting guidance developed at EU level to the relevant national or local context and that help assessing the effectiveness of any such measures. The EUCPN should further deepen the evidence base for crime prevention measures but also explore ways to facilitate the implementation of crime prevention measures at local level, while staying up to date and being responsive to new developments. This can be achieved in different ways, including through:

-Cooperation with other relevant organisations and EU initiatives, projects and networks, in order to combine insights and create synergies when developing and disseminating knowledge and best practices;

-Enhanced cooperation with relevant networks or crime prevention councils at national level, in order to improve the EUCPN’s outreach to target groups;

-An increased focus on responding to new developments and converting research products into practical knowledge and tools, geared towards practitioners’ needs, both in terms of content and form (including making available budget for translations).

The EUCPN plays an increasing role in contributing to the development of EU policies in the field of crime prevention, in particular through its involvement in EMPACT. The Network’s contribution to the various aspects of crime prevention at the EU level could be improved by:

-Deepening working relationships within EMPACT -notably with EMPACT drivers and EU agencies in order to broaden its active contribution to embedding crime prevention measures in operational actions.

-Establishing closer links between the National Representatives and the National EMPACT coordinators, so as to ensure coherence in the Network’s contribution of EMPACT and enhance awareness of EMPACT procedures and working methods;

-Fostering its cooperation with the European Commission, to ensure alignment with current and future EU policies, strategies, networks and projects.

1.    Lead DG, Decide Planning/CWP references

The Evaluation Roadmap for the initiative was published by DG HOME on the Commission’s ‘Have your say’ webpage 55 in February 2019. The Terms of Reference were drawn up for engaging a contractor to carry out the external study as part of the evaluation. A request for service was issued on 27 January 2019, and a contractor selected by an evaluation committee. The study began in March 2019 and ended in December 2020. The agenda planning (Decide) reference assigned to the evaluation is PLAN/2021/13100.

2.    Organisation and timing

As per the Better Regulation Guidelines, an inter-service steering group was used within the Commission to oversee the evaluation. The Secretariat General, DG JUST and the Legal Service were invited to nominate representatives to the steering group.

The meetings of the steering group were chaired by DG HOME. The steering group was regularly consulted over the course of the external evaluation, typically in conjunction with the submission of specific draft reports by the contractor responsible for carrying out the external study. These consultations took place both in the context of regular meetings, via email and telephone.

3.    Exceptions to the better regulation guidelines

In conducting the evaluation, no exceptions from the usual procedural requirements described in the Better Regulation Guidelines were required.

4.    Evidence and sources 

The evaluation drew on different types of documents. Firstly, the external contractor examined EU policy and legislative documents and guidelines, in order to provide the overarching policy framework to be considered when assessing the coherence of the Network and its actions, and to assist in designing recommendations for the future. In order to provide the evidence base for the analysis of the Network’s objectives and tasks since 2012, EUCPN outputs were examined. Outputs include the Multi-Annual Strategies and Annual Work Programmes, but also the Reports, Toolboxes, Crime Monitors and newsletters. In order to complete the information on the Network, various back-office documents were included in the list of sources. These documents include meeting minutes, reports on activities written by the Secretariat for the members of the EUCPN, financial documents and stock-taking documents on the progress of actions under the Multi-Annual Strategy, among others.

   The list of sources also includes relevant studies and reports, including scientific literature focusing on the current state of play of crime prevention in Europe. The previous evaluation of the Network in 2012 was also consulted. Finally, national and international measures on crime prevention, including legislative acts, standards and procedures, were consulted, in order to gain insight on the extent to which EU Member States have benefitted from the outputs of the Network.

Aside from a review of the relevant documents, the evaluation also relied on extensive consultations with a wide range of stakeholders. These consultations served as opportunities to collect new data or to confirm the validity of already collected data. Additional information on the stakeholder consultations is provided in Annex V.

1.    Methodology

The external evaluation study was carried out by Ernst & Young (‘EY’) following a call for services under a framework contract. The evaluation was conducted through a mixed methods approach and was informed by the triangulation of a variety of sources. A range of methodological tools and techniques were used.

The evaluation study included a broad desk research that focused on a review of all relevant studies and literature. It covered EU policy and legislative documents and guidelines, EUCPN outputs, EUCPN back-office documents, relevant studies and reports, including scientific literature, and national and international measures on crime prevention. In order to identify additional information that could contribute to framing and explaining the evidence collected on the performance of the EUCPN, the Team sought to understand how other European networks operate, with the aim of identifying what works best and what could be done differently. 56

A wide range of stakeholders were consulted as part of the external study. The consultations included an online survey with stakeholders, targeted interviews and EUCPN meeting attendance (see Annex V for the synopsis report of the consultation study), and a public consultation carried out by the European Commission. Furthermore, EY carried out four case studies within the scope of selected actions within the Multi-Annual Strategy 2016-2020, to examine more in depth the key aspects of the evaluation criteria:

-the promotion of EUCPN output in Member States’ crime prevention campaigns and Projects

-collection and dissemination of methodologies to assess the impact of crime prevention

-the alignment of EUCPN’s crime prevention activities to EMPACT

-insights on the added value of EUCPN support to Member States

Furthermore, the contractor carried out a traffic light assessment of all 25 actions foreseen in the EUCPN’s Multi-Annual Strategy 2016-2020.

2.    Timing 

While the Evaluation Roadmap, which was published in February 2019, indicated that the external evaluation should have been completed in the first quarter of 2020, the actual completion date was in the second quarter of 2021. In order to allow enough time to process and analyse the outcomes, the Commission opted to extend the evaluation timeframe into the second quarter. As a result of the COVID19 pandemic, the contractor and the Commission agreed to extend the duration of the supporting study. In order to mitigate the temporal gap between the conclusion of the external study and this evaluation, developments after the completion date of the external study have been included in this evaluation, when relevant for the assessment of the development of the Network. The descriptions of the most recent developments are based on fact finding by the Commission, based on open sources and interviews with the EUCPN Secretariat.

3.    Limitations 

The external study encountered some difficulties when performing the desk research, most of which have been addressed with adequate mitigation measures.

In particular, difficulties relate to:

-Information gaps on EUCPN: it was challenging to retrieve enough elements to evaluate efficiency.

oSolution: the contractor reached out to the EUCPN Secretariat to request further documentation (i.e. budgetary documents) and asked for documentation that would help addressing the evaluation questions on efficiency. Despite this, it was not possible for the contractor to measure the hours/days spent on each MAS goal and/or MAS action or compare Full-Time Equivalents (FTE) dedicate to communication activities.

-Lack of consistency in EUCPN documentation: some of the information needed to address the judgement criteria was available but scattered across different types of documents (e.g. number of participants to meetings, seminars and the Best Practice Conference (BPC) and European Crime Prevention Award (ECPA)), to the point that it was not possible to reconstruct the full picture.

oSolution: The Team collected the information available across all documents into partial tables and requested the Secretariat to fill the data gaps. The Secretariat was able to fill the information gaps only partially, as it does not collect some of the information needed do address the judgement criteria (e.g. N. of participants to BPC-ECPA, number of downloads from the EUCPN website).

-Not all indicators assumed to be relevant to meet the judgement criteria were adequate: As mentioned above, some of the information we expected to be able to retrieve from the Network in order to address the judgement criteria was not as relevant as expected.

oSolution: the contractor had to discard some indicators.

-Limited comparability of information found at the national level: The variety of source types identified at the national level related to crime prevention strategies (e.g. national laws, institutional websites) resulted in heterogeneous information with a low degree of direct comparability.

oSolution: The contractor developed a matrix of convergence structured around key areas of analysis in order to limit differences and ensure a common minimum ground of comparability. This template was slightly adjusted while progressing with the desk research in order to ensure its relevance to available information.

The study encountered some difficulties when performing the consultations, most of which have been addressed with adequate mitigation measures.

-National Representatives: Not all NRs were able to provide informative input to our questions. As a consequence of the high turnover experienced by the EUCPN Board, several NRs or substitutes were relatively new to the EUCPN at the time of the interview and were not knowledgeable enough about the Network over the period covered by this study.

oSolution: While NRs remained the primary focus target of our stakeholder consultation strategy, the contractor devised some mitigation measures to compensate for the information gaps.

§The contractor acted to identify, contact and interview knowledgeable stakeholders, formerly involved with the EUCPN, that could fill in the information gaps.

§To gain a broader understanding of the Network as a whole, the contractor interviewed the former head of the Secretariat and the crime prevention expert who developed the Standard Conceptual Framework for the Description and Exchange of Good Practices for the EUCPN.

§The contractor decided to conduct a group interview with the Secretariat to gain specific information on questions that NRs were not able to answer.

-Stakeholder classification: When asked to present themselves, several interviewees fit more than one of the stakeholder categories identified within the scope of this study. This can be in part attributed to the multidisciplinary nature of crime prevention. In two occasions interviewees were not confident to say whether they could be considered NCPs of the Network, as their services have not been requested in years.

oSolution: While the original plan was to always use specific interview guidelines for every type of stakeholder, when facing an interviewee fitting multiple profiles, the contractor opted for extending the questioning, in view of retrieving the information the interviewee was knowledgeable about to the maximum extent possible.

-Unresponsive organisations for the benchmarking exercise: Despite multiple attempts to contact different members of these organisations, the Team did not receive any response.

oSolution: the contractor agreed to conduct desk-based research on relevant European networks instead.

-Limited responses to the public consultation: the public consultation only received two responses.

oSolution: the public consultation was deemed not to be an adequate basis for analysis.

As for the collection of information through the online survey:

-Survey refinement and consequent delay: the first draft of the questionnaire was not exhaustive enough in presenting the object of the inquiry, especially for people unfamiliar with the Network.

oSolution: To make sure the questionnaire would collect meaningful responses from the stakeholders involved, the team reviewed extensively the questionnaire to prevent 1) inaccurate answers stemming from unclear questions 2) lower response rate resulting from ‘drop-outs’ who abandon the survey mid-way through.

-Survey fatigue: Several stakeholders approached by the contractor, either in person or via email, told us they had already been interviewed.

oSolution: The contractor included a specific alert in the presentation email to distinguish the survey from the others and sent targeted reminders that significantly increased the number of responses.

Despite the difficulties encountered, the quality of the information gathered is generally satisfactory thanks to a combination of additional actions taken on the part of the contractor and the role played by stakeholders in validating the information that was collected. The information is satisfactory both in terms of quality and of breadth of representation from different categories of stakeholders, and Member State representation.

4.    Commission assessment of the work of the contractor

Overall, the Commission is satisfied with the work of the external contractor, and the solutions adopted when faced with limitations. Following the external study, the Commission chose to simplify and streamline its work, in order to get a more targeted set of conclusions and recommendations. The Commission opted to only use Council Decision 2009/902/JHA as the basis for the Network’s mission and objectives and regarded the MAS 2016-2020 as strategic document that sets out to achieve these goals. This led the Commission to simplify the intervention logic and base the evaluation on the four identified objectives and results set out.