EUROPEAN COMMISSION

Brussels, 22.3.2022

COM(2022) 121 final

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

assessing the implementation and achievements of the 2014-2020 justice programme

{SWD(2022) 64 final} - {SWD(2022) 69 final}

The Treaty on the Functioning of the European Union provides for the creation of a European area of freedom, security and justice based on the mutual recognition of judicial decisions and mutual trust among Member States. Within this area, people are free to move and can rely on the respect of fundamental rights, as well as on shared principles, such as non-discrimination, gender equality, effective access to justice for all, the rule of law and well-functioning independent judicial systems.

These legally binding provisions in the justice area, set by the Treaty, were also reaffirmed by the European Council in the Stockholm programme. 1 The achievement of a Europe of law and justice is one of the political priorities of the EU and the 2014-2020 justice programme is one of the instruments that contribute to achieving this objective.

This report sets out the results obtained so far by the 2014-2020 justice programme and the qualitative and quantitative aspects of its implementation, as required by Article 14(2)(c) of the regulation that set up the justice programme for the 2014-2020 period ( ‘the Regulation’). 2  The report does not assess the long-term impact and the sustainability of the effects of the justice programme. This limitation is due to the fact that around 30 % of the projects funded by the justice programme are still ongoing. This situation has been exacerbated by the COVID-19 pandemic, which has caused the extension of the duration of several projects

For these reasons, and in order to have a meaningful evaluation of the programme’s long-term results and impacts, the ex-post evaluation will be carried out in two parts. This report and its supporting documents, represent the first part of the evaluation. It is based on the currently available data, provides an overview of the funding distribution and assesses the achievements of the justice programme so far. While the new justice programme 2021-2027 was already adopted, based amongst others on the results of the interim evaluation of the justice programme 2014-2020, this first part of the ex-post evaluation of the previous programme will be of added value in order to shed light on potential areas for improvement in the implementation phase of the new justice programme.

The second part of the evaluation will be carried out at a later stage, once all final data will be available and in concomitance with the interim evaluation of the succeeding justice programme (2021-2027). This second part will assess the long-term impacts and sustainability of the effects of the justice programme and will provide recommendations for the new Multiannual Financial Framework after 2027, as appropriate. 

This report is based on the findings of the evaluation prepared by the European Commission 3 and supported by an external study. 4

1.Introduction and background

The 2014-2020 justice programme was set up by Regulation (EU) No 1382/2013 of the European Parliament and Council. Prior to 2014, three individual programmes, the criminal justice programme, the civil justice programme and the drug prevention and information programme, covered the same thematic areas addressed by the justice programme.

The Regulation sets its general objective as being to:

·contribute to the further development of a European area of justice based on mutual recognition and mutual trust, in particular by promoting judicial cooperation in civil and criminal matters.

Its specific objectives are to:

·facilitate and support judicial cooperation in civil and criminal matters;

·support and promote judicial training for professionals (such as judges, prosecutors, notaries, prison staff and lawyers) on civil and criminal law EU instruments, fundamental rights, judicial ethics and the rule of law, including language training on legal terminology, with a view to supporting a common legal and judicial culture;

·facilitate effective access to justice for all, including to promote and support the rights of victims of crime as well as the procedural rights of suspects and accused persons in criminal proceedings;

·promote initiatives in the field of drug policy in relation to the judicial cooperation and crime prevention aspects, insofar as these kinds of initiatives are not covered by the Internal Security Fund for financial support for police cooperation, preventing and combating crime, and crisis management or by the health for growth programme. 5

The programme is implemented by the European Commission via direct management.

In terms of geographical coverage, the justice programme is open to all EU Member States with the exception of Denmark and the United Kingdom. 6 In addition to the EU Member States, the European Free Trade Association States that are party to the European Economic Area, candidate countries, potential candidates and countries acceding to the EU were also eligible to participate in the programme, provided that they concluded an agreement with the Union laying down the details of their respective participation in the programme. Albania joined the justice programme as of 2017 and Montenegro as of 2019.

2.Key elements and implementation of the programme

In line with Article 6 of the Regulation, the justice programme finances a wide range of activities such as analytical activities, mutual learning, cooperation, awareness raising and dissemination activities, training activities and actions to support the main actors, i.e. European networks, public and private organisations, which are usually non-profit-oriented, national, regional and local authorities in EU Member States, civil society organisations; universities, research institutions, and international organisations, whose activities contribute to its specific objectives.

Target groups, (groups that can benefit, either directly, by participating in justice projects' activities, or indirectly from the programme’s implementation), include potentially all people in the EU, since the programme aims to create a European area of justice where everybody is aware of and can exercise their rights.

As provided for by the Regulation, the programme uses action grants, operating grants and procurement actions as the main funding mechanisms to support projects related with its objectives.

2.1. The programme’s specific objectives

·Specific objective 1: Judicial cooperation

The programme supports activities that contribute to the effective and coherent application of the EU acquis relating to judicial cooperation in civil and criminal matters, also by building-up and/or improving data collection and statistics on the application of the EU acquis. Funded activities also contribute to the enforcement of EU instruments and judicial decisions, in particular resulting from cross-border disputes. The programme also finances projects aiming to improve the exchange of information among professionals in order to enhance the operational cooperation and mutual trust in the EU.

·Specific objective 2: Judicial training

The programme supports projects that encourage the training of justice professionals on EU law, including language training on legal terminology with a view to supporting a common legal and judicial culture in the EU. Judicial training can address ’general’ competences, such as linguistic skills and terminology, as well as more specialised aspects, such as knowledge of EU law, judicial cooperation in civil and criminal matters. It can take various forms such as seminars, e-learning, exchanges of professionals, development of good-practice guides and sharing of experience. Activities funded mainly support training for the members of the judiciary and judicial staff, but also cover other justice practitioners associated with the judiciary. Finally, the development of tools for training providers is also promoted. The ultimate goal is supporting the correct application of EU law through the exchange of best practice and networking.

·Objective 3: Access to justice

Actions financed in this area aim to provide EU citizens with effective remedies when EU law has been broken, especially where national procedures are too difficult for citizens to be used. In particular, the programme promotes the use of alternative types of remedies and non-remedies developed in the EU, which can provide a quick, efficient and less costly solution to disputes, for example, those supported by the e-justice portal. The activities funded also aim to encourage a close cooperation between national authorities and administrative bodies, which is particularly important for the effectiveness of certain EU rights.

·Objective 4: Drug prevention policy

In the area of drug prevention policy, the programme promotes initiatives that focus on judicial cooperation and crime prevention. The main priorities are to promote the practical application of drug-related research, support civil society organisations and key stakeholders, expand the knowledge base and develop innovative methods to address the phenomenon of new psychoactive substances.

2.2. Budget

The initial total programme's budget for 2014-2020 was EUR 377 604 000. According to the annual work programmes, a total of EUR 333 971 000 has been planned between 2014 and 2020.

The tables below summarise the distribution of the budgetary planning per year and across specific objectives.

Table 1 Budget planned per year

Source: 2014-2020 annual work programmes

The overall budget increased steadily up to 2017, when it reached a peak of EUR 52 631 000. It declined from then until reaching approximately the same level as in 2014. The budget decreased as funds of the justice programme were reallocated to the Civil Protection Mechanism (CPM - COM(2017)772) and to provide eu-LISA with the necessary financial resources to develop the European Criminal Records System for third country nationals.

Table 2 Budget planned per specific objective 2014 -2020 (in €)

Source: annual work programmes 2014-2020

The distribution of funds across specific objectives shows that, in every year, the specific objective ‘judicial training’ received a large part of the available financial resources. As a result, the programme complies with the allocation set out in the annex to the programme’s legal basis. Judicial cooperation and access to justice are also important pillars of the justice programme. As for drug prevention policy, when considering the low proportion of the budget, account should be taken of the fact that the focus of activities in this field mainly lies in complementing initiatives related to judicial cooperation and crime prevention. Furthermore, this specific objective is highly correlated with other EU instruments such as Internal Security Fund or the health for growth programme.

2.3. Applications received and projects selected

In the period covered by the 2014-2020 annual work programmes, 118 calls for proposals were launched. A large proportion of projects was funded under the specific objective access to ‘justice for all’ (JACC).

The tables below summarise the allocation of funds and the distribution of types of activities implemented through projects.

Table 3: Number of funded projects per year (Action Grants (AG) and Operating Grants (OG)

Source: Data for 2014-2015 are retrieved from the interim evaluation of the justice programme. The projects funded for 2016-2020 have been retrieved from the project database

Overall, the number of projects funded decreased during the programming period, reaching its lowest point in 2020 with 49 projects. This decrease is generally reflected within the specific objectives, which also showed an overall decrease in the number of projects selected for funding.

Regarding the type of activities, implemented through action and operating grants, Figure 1 below depicts the distribution of activities over the entire programming period. 7  Training activities and analytical and monitoring activities were key components of projects funded.

Figure 1 Types of activities implemented by projects (AGs and OGs) funded by the justice programme (2014-2020)

Procurement was also a significant funding mechanism throughout the programming period. A total of EUR 81,7 million was allocated for procurement in the 2014-2020 annual work programmes. The largest proportion of the budget for procurement activities was allocated to the ‘judicial training’ specific objective (36%), followed by ‘access to justice’ (30%). 

Table 4 Number of procurement contracts signed per year

Information communication technology tools and other IT-related activities were the most common types of  procurement activities contracted (see Figure 2) over 2014-2020, followed by mutual learning.  11

Figure 2 Types of procurement activities (2014-2020)

Source: analysis of procurement data provided by DG JUST.

2.4. Main achievements of the programme

The system of indicators introduced for the 2014-2020 justice programme has proven to be appropriate for measuring its achievements.

One cross-programme indicator measures progress towards the general objective of creating a European area of justice based on mutual recognition and mutual trust. The indicator is defined as the cumulative number of legal professionals receiving training. All EU judicial training activities are included, not only those through the justice programme. The target to be reached by 2020 is set at 700 000 legal practitioners receiving training. This was reached well in advance, in 2017, with 820 199 legal practitioners trained.

In general, the indicators appropriately reflect the priorities for each specific objective in the programme, and are measurable with respect to their baseline. The evaluation carried out has shown significant progress on several fronts. For example, in the area of judicial cooperation in civil and criminal matters, the number of exchanges of information in the European Criminal Records Information System had surpassed its 2020 target of 3 500 000 exchanges in 2019. In relation to the ‘access to justice’ specific objective, both 2020 targets, the number of hits on the e-justice portal and the number of victim support organisations with national coverage have already been reached or exceeded. 12  A similar success was recorded in the field of judicial training, where the flagship target of the 2011 – 2020 European judicial training strategy 13 , to train half of all legal practitioners (800 000) in EU law between 2011 and 2020, was met in 2017. Additionally, the programme contributed to the achievement of the judicial training strategy’s overall goal that EU funds support the training of at least 20 000 legal practitioners annually. Within the framework of the judicial training strategy, the justice programme’s initial annual goal of 16 000 members of the judiciary and judicial staff trained was met in 2015 and 2017, as a subsequent reduction in funds for judicial training calls for proposals led to a reduction in the number of people trained.

However, some of the selected indicators are difficult to measure. A case in point is the second indicator for judicial cooperation, which is defined as ‘the average length of the surrender procedure under the European Arrest Warrant’. This indicator, which did not reach its target, is difficult to measure since it is influenced by external factors. These include for example the different degree of incorporation of EU directives at national level, as well as the different levels of judicial system reforms within EU Member States, which have a significant impact on the level of enforcement of the EU acquis. 14  Another example is the indicators related to drug prevention policy. Both indicators the number of new psychoactive substances researched and the number of problem opioid users in drug treatment did not reach their target. In contrast to the external factors that impacted the surrender procedure under the European Arrest Warrant, the indicators related to drug prevention policy were more influenced by macroeconomic conditions.

As a result, due to the influence of external factors, it is difficult to assess the extent to which the programme contributed to meeting the targets set by selected indicators.

There is still room for further improvements as there is currently no uniform satisfaction survey to capture the perceptions of participants, who took part in training activities. Each project implemented its own evaluation scheme. However, as of 2021, the programme collects data from beneficiaries in relation to the perceived quality of funded activities in a more consistent way.

2.5.     Participants and partnerships

The results from the analysis strongly suggest that the justice programme creates successfully transnational partnerships. The programme's transnational nature is a key part of its unique added value, according to responses from the Justice Programme Committee Members from EU Member States.

According to beneficiaries, partnerships developed through operating grants and action grants under the justice programme have had beneficial effects on the capabilities of their respective organisations. Beneficiaries report substantially increasing their organisation’s capacities by benefitting from networks and mutual learning experiences funded through the programme. Operating grants promoting the creation and maintenance of networks are essential pillars that support actors in developing judicial cooperation, access to justice and judicial training. Operating grants are of key importance, as they provide framework partners with the possibility to plan their activities over the course of four years with the European Commission's support.

While networks contribute to the transfer of knowledge and capacities across EU Member States, it is clear that Member States did not participate equally in the justice programme. In terms of geographical distribution of resources, participation in the programme was more concentrated in a number of Member States (e.g. Italy, Belgium, Spain, Germany, the Netherlands and France) while for instance there was a lower demand from Member States, which joined the EU after 2004, except for Bulgaria and Romania. About 22% of all beneficiaries are networks established either in either Italy or Belgium who carry out activities that cover most of the EU Member States.

3.Results of the evaluation

As demonstrated by the study, the justice programme is performing well with regard to its specific objectives, in terms of effectiveness, efficiency, relevance, coherence, complementarity, and synergies, EU added value, equity and scope for simplification. There is room for improvement, in particular, in relation to the programme’s geographical distribution of resources.

3.1. Effectiveness

In terms of effectiveness 15 , the current analysis confirmed the main result of the interim evaluation. The activities funded through action grants, operating grants and procurement achieve the expected results. In other words, the budget available has been used to achieve results that are well aligned with the programme’s general and specific objectives.

While the programme-specific indicators are appropriate for monitoring progress towards the programme’s objectives, the programme’s current monitoring system could benefit from a more systematic and robust approach to data collection. The Commission addressed this problem in its proposal leading to Regulation 2021/693 providing the legal base for the 2021-2027 justice programme. As a result, from 2021, the justice programme will address the need to obtain more in-depth insights about target groups, which was identified as a potential area for improvement for the 2014-2020 programme. Moreover, since the programme has a wide scope, its effectiveness is affected by internal and external factors. The main external factor mentioned by beneficiaries in interviews is the COVID-19 pandemic, which required many beneficiaries to re-design the implementation of their project or to cancel it. However, macroeconomic conditions, the refugee crisis and the transposition of EU law by Member States are also external factors that affected the justice programme’s effectiveness. As for internal factors, the most salient area for improvement is the application procedure. The study shed light on the need to provide more guidance and assistance to applicants, in addition to the existing materials. This would allow the playing field among applicants to be more level and thereby broaden the pool of quality proposals. This in turn, would increase the programme’s effectiveness.

3.2. Efficiency

The programme made also significant progress in terms of its efficiency compared to its predecessor programmes. 16  Consulted stakeholders who were consulted repeatedly confirmed the cost efficiency of the programme. Applicants and beneficiaries alike stressed that the relatively high costs of drafting a proposal are outweighed by the even higher benefits of participating in the programme. This finding is in line with the positive feedback on the streamlining of the application process. However, there are also some aspects on which more critical views were expressed. For example, applicants repeatedly stated that despite visible improvements in the management cycle, they had experienced difficulties in complying with the application requirements in the time allotted by the European Commission. In relation to the use of the available budget by project beneficiaries, underspending at project level still impacts negatively the programme’s efficiency.

3.3. Relevance

The analysis showed very high levels of relevance 17  for the programme and its actions. The analysis concludes that project objectives and activities are relevant to the needs of targeted stakeholders. Vice-versa, the groups targeted through the programme are relevant for the achievement of the programmes’ specific objectives. This is the result of the programme’s structure, which is flexible enough to adapt and modify priorities to emerging needs on the ground. However, a few areas for improvement were identified.
These include engaging more systematically and openly with stakeholders to better align the activities and specific objectives outlined in the annual work programmes with the needs of the programme’s target groups.
There is also a need to continue existing and successful efforts to expand the programme’s target group and include more stakeholders relevant to the programme. In particular, civil society organisations in the area of victims’ rights, could be even more actively supported.

3.4. Coherence, complementarity, synergies

The current analysis found that the programme has a good level of coherence and complementarity with other EU instruments, programmes and actions. It underscored that, especially in the case of judicial training, the merging of the predecessor programmes into the justice programme has increased the coherence with other EU initiatives. The same is true for other training objectives, as it reduced the likelihood of duplications. This result confirms the findings outlined in the interim evaluation. Complementarity with national projects is also high. This has been achieved by the transnational design of activities funded under the justice programme, which ensures that national activities do not replace or replicate funded activities funded under the justice programme. The programme is also coherent with international obligations, such as the UN 2030 sustainable development agenda. For instance, the UN objective of ensuring peace and security is linked to the creation of just and inclusive societies that provide equal access to justice, effective rule of law, and transparent and effective judicial institutions.

The programme also contributes to the harmonisation of international private law by financing the EU’s membership in The Hague Conference on Private International Law.

Nevertheless, as potential for overlaps exists, this should be closely monitored to maintain the high level of complementarity.

Finally, programme beneficiaries have identified and maximised synergies with projects funded by other EU initiatives, such as the rights equality and citizen’s programme, as well as with other projects within the justice programme. However, the exchange of expertise and best practices between different beneficiaries is an area for improvement. Thus, there is untapped potential for beneficiaries to make use of the networking opportunities the justice programme offers and actively engage in dialogue with each other.

3.5. EU added value

The study fully confirms the programme's high added value. 18 In particular, the EU added value is evident through the promotion of transnational projects to tackle cross-border issues, especially in training and information exchanges. High EU added value is also created by the provision of crucial financial resources to fund activities in key areas that are not necessarily high on the agenda of individual Member States. In addition, some projects (like exchanges of professionals) by their nature require joint actions of all Member States. Thus, individual action by Member States would not be able to achieve the same results in terms of size and scope as the justice programme. This finding is supported by the results of the Justice Programme Committee member survey which show that the majority of members believe that without EU funding, the activities implemented through the justice programme would not be possible.

Against this backdrop, the conclusion drawn in the interim evaluation of the programme still holds true: limiting EU intervention would have negative effects on the overall capacity of national and European actors to address issues in the areas covered by the programme or implement the types of activities funded by the programme. 19  The most likely consequences of a reduction or withdrawal of EU involvement through the justice programme would entail significant difficulties in developing transnational projects. These would be accompanied by reduced coherence across Member States in terms of understanding and implementation of EU law.

This would also undermine the adoption of innovative legislative frameworks and the introduction of cutting-edge approaches to tackling the needs of vulnerable groups and developing new tools and practices. In other words, it would be difficult to achieve objectives responding to an EU-wide strategy.

3.6. Equity

With a view to the equity 20  of the programme, the current evaluation reiterates the finding from the interim evaluation that resources and support provided through the programme are not distributed in a balanced manner across different target groups, EU Member States and types of beneficiaries. On the macro level, there is a discrepancy between high application rates from countries like Belgium, Italy, Spain, Germany, the Netherlands and France and comparatively low application rates from countries like Latvia, Poland, Croatia, Slovakia and Cyprus. An improved geographical spread of resources across participating countries is, therefore, a key area for improvement.
On the meso level, the analysis revealed that overall the programme’s target groups have become more diverse and include a wider variety of justice professionals. Yet, as the legal base for the 2021-2027 justice programme highlights, it will be important that civil society organisations are more involved in interventions that the programme funds relating to the promotion of victim’s rights. This finding dovetails with the low application rates in the aforementioned countries. Civil society organisations from underrepresented countries might need additional support as they might not have the necessary capacity to apply to the justice programme.
On the micro level, more robust and systematic monitoring data would allow for deeper insights into the target group’s composition to be drawn. This could be facilitated by collecting data, that are disaggregated inter alia by sex, age and disability status. In turn, the insights gathered at project level could inform the programming on a macro level to increase the funding’s impact on an even more diverse group of stakeholders. This finding is also related to the cross-sectional themes like gender equality, children’s rights and the rights of people with a disability. More robust data would therefore facilitate the identification of areas, in which the rights of women, men, girls and boys and people with a disability are not as well promoted as they could be.
The promotion of the cross-sectional priorities of gender equality, rights of the child and rights of people with disabilities is highly significant for the justice programme and it is enshrined in its legal basis. In particular, the principles of mainstreaming gender equality and children’s rights are evaluated during the process, under the quality of proposals. While these cross-sectional priorities are deeply embedded within the programme, the analysis concludes that some applicants struggle with understanding these concepts. As a result, some projects funded through the programme do not live up to their potential in promoting gender equality, the rights of the child and the rights of people with disabilities. Although all cross-sectional themes require further attention, there are also differences in the degree to which the programme addresses them at project level. While gender equality and children’s rights receive specific attention in funded projects, the rights of people with disabilities are not as prominently featured within projects. Thus, there is a need to collect better data on the composition of targets and to raise more awareness among applicants on cross-sectional themes such as gender mainstreaming, children’s right and rights of people with a disability.

3.7. Scope for simplification

Concerning the scope for simplification 21 , it should be noted that important changes have been adopted by the European Commission to review, speed up and simplify the process during the 2014-2020 programming period. However, while the current management model is considered to be appropriate, and changes have been positively welcomed by project beneficiaries, beneficiaries have identified some issues and room for further improvement.

The scope for simplification includes the need for simpler application and results-based reporting procedures, improved monitoring and management tools to more systematically collect information on outputs, results and targets achieved, as well as help for organisations with difficulties in working in English only. However, it should be noted that the perceived burden and complexity varies among project beneficiaries. Organisations with larger capacity and greater experience in applying, coordinating, and participating in projects report a much easier and smoother situation than smaller organisations and first-time applicants.
Finally, despite improvements in the monitoring and management tools, there is still room for improvement, especially when it comes to adapting the application portal to the types of beneficiaries supported by the programme or supporting first-time applicants and beneficiaries to navigate the tool without the risk of delaying their activities.

4. Conclusions and the way forward

In conclusion, the justice programme is a unique financial instrument in the area of justice and rights in Europe. Its resilient design allows it to respond to external shocks, such as the COVID-19 pandemic and find flexible answers to emerging needs and problems. This report on the first part of the ex-post evaluation of the 2014-2020 justice programme confirms it is contributing to upholding EU values, such as the rule of law, the independence of the judiciary and the effectiveness of the justice system. The report also underlines the programme’s key role in supporting Member States to achieve more efficient justice systems. In fact, the interviews conducted with stakeholders clearly indicate the importance of the programme for the purpose referred to in the text. While the long term impacts are not known, short term results are visible for example in terms of exchanges of best practices and digitalization of justice systems. The 2014-2020 programme has already proven to be an important tool to establish a European area of justice based on mutual recognition and mutual trust, yet, there are still areas for improvement for future justice programmes, as outlined in this report:

·a closer alignment of the timing of the application cycle with stakeholders’ needs;

·the implementation of a better monitoring system to more systematically collect information on outputs, results and target groups;

· targeted activities ensuring a more balanced geographical spread of funds across participating countries;

·finally, the further integration of results-based reporting and dedicated initiatives to facilitate the application process for applicants, who currently lack the capacity, expertise and experience to draft quality proposals, including with a view to designing non-underspending projects, might contribute to even more effective, efficient, relevant, equitable and accessible justice programmes in the future.

Bearing these opportunities in mind, the current justice programme as of 2021, already addressed some of the identified points, for example, by introducing a more granular monitoring of project data to provide a better insight into the project outputs, results and target groups.

(1)

OJ C 115, 4.5.2010, p. 1.

(2)

 Regulation (EU) No 1382/2013 of the European Parliament and of the Council of 17 December 2013 establishing a justice programme for the period 2014 to 2020 (OJ L 354 of 28.12.2013).

(3)

Commission staff working document accompanying the report on the ex-post evaluation of the implementation of the Justice programme 2014-2020.

(4)

 Ex-post evaluation of the justice programme 2014-2020, Fondazione Giacomo Brodolini Srl SB and Ernst & Young, Final Report, August 2021.

(5)

For more info, see https://ec.europa.eu/home-affairs/financing/fundings/security-and-safeguarding-liberties/internal-security-fund-police_en and https://ec.europa.eu/health/funding/programme/2014-2020_en .

(6)

 The justice programme has legal bases that belong to Part V of title III of the TFEU. Therefore, protocols 21 and 22 to the TEU and the TFEU apply and Denmark and the UK never participated to the justice programme.

(7)

Multiple activities per project were considered when creating the overview.
To see the types of activities funded by the Justice Programme in 2014, 2015 and 2016, see the interim evaluation of the Justice Programme 2014-2020, Ernst & Young Financial-Business Advisors, final report, April 2018, p. 71-72.

(8)

For 2017, one procurement contract was not associated with a specific objective.

(9)

For 2018, two procurement contracts were not associated with a specific objective.

(10)

For 2019, two procurement contracts were not associated with a specific objective.

(11)

The average is calculated over the procurement contracts provided by DG JUST, which, for 2014 and 2016, is lower than the numbers included in the interim evaluation.

(12)

The e-justice portal provides information on justice systems and improves and facilitates the access to justice throughout the EU, in 23 languages. More info available at the following link: https://e-justice.europa.eu/home .

(13)

COM(2011)551 final.

(14)

These obstacles were mentioned by several projects and shown also by the ‘justice scorecards’.

(15)

 Effectiveness: whether and to what extent the Justice Programme has achieved its general objective, as well as its four specific objectives, and which are the factors that have contributed to these achievements.

(16)

 Efficiency: whether and to what extent the costs of the programme were proportionate given the benefits achieved and which parameters/factors participated in these results.

(17)

 Relevance: whether and to what extent the justice programme addresses needs and problems of the target groups identified in the 2011 impact assessment and in the legal basis of the programme (as well as emergent needs related to the creation of a European Area of Justice) and whether its objectives are still relevant for the needs and problems of the beneficiaries.

(18)

EU added-value: to what extent the effects from EU action are additional to the value that would be the likely consequence of limiting the EU intervention through the programme.

(19)

 Finding also supported by interview feedback from five beneficiaries.

(20)

Equity: whether and to what extent the Justice Programme has distributed the available resources fairly among beneficiaries in different Member States, took into consideration the needs of target groups, promoted gender mainstreaming, the rights of the child and the rights of people with disabilities.

(21)

 Scope for simplification: whether and to what extent the management of the justice programme could be further simplified.