EUROPEAN COMMISSION
Brussels, 3.6.2025
SWD(2025) 133 final
COMMISSION STAFF WORKING DOCUMENT
EVALUATION
Accompanying the document
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF REGIONS
on the ex post evaluation of the Rights, Equality, Citizenship (REC) programme (second part), the ex post evaluation of the Europe for Citizens (EfC) programme and the interim evaluation of the Citizens, Equality, Rights and Values (CERV) programme
{COM(2025) 266 final}
Table of contents
1.Introduction
1.1.Purpose and scope of the evaluation
1.2.Methodology framework and data limitations
2.What was the expected outcome of the intervention?
2.1.Description of the interventions and their objectives
2.2.Intervention logics
2.3.Points of comparison
3.How has the situation evolved over the evaluation period?
3.1.Rights, Equality and Citizenship implementation
3.2.Europe for Citizens implementation
3.3.Citizens, Equality, Rights and Values implementation
4.Evaluation findings (analytical part)
4.1.To what extent was the intervention successful and why?
4.1.1.Effectiveness
4.1.2.Efficiency
4.1.3.Coherence
4.2.How did the EU intervention make a difference and to whom?
4.3.Is the intervention still relevant?
5.What are the conclusions and lessons learned?
5.1.Rights, Equality and Citizenship programme
5.2.Europe for Citizens programme
5.3.Citizens, Equality, Rights and Values programme
5.4.CERV: operational findings and lessons learned
Annex I. Procedural Information
Annex II. Methodology and Analytical models used
Annex III. Evaluation matrix
Annex IV. Overview of benefits and costs, simplification and burden reduction
Annex V. Stakeholders consultation - Synopsis report
Annex VI: State of play in the policy areas of CERV
Table of abbreviations
1.Introduction
This Staff Working Document (SWD) presents the results of the ex post evaluation of the Rights, Equality and Citizenship (REC) programme, the ex post evaluation of the Europe for Citizens (EfC) programme as well as the interim evaluation of the Citizens, Equality, Rights and Values (CERV) programme. The evaluation is subject to a report to the European Parliament, the Council, the European Economic and Social Committee, and the Committee of the Regions.
1.1.Purpose and scope of the evaluation
Two successive generations of EU programmes have furthered the goal of protecting and promoting the rights and values enshrined in the EU Treaties and the Charter of Fundamental Rights of the European Union: the 2014-2020 REC and EfC programmes, and the 2021-2027 CERV programme. Taking over the baton from its two predecessors, the CERV programme builds upon the results they delivered.
The
Rights, Equality and Citizenship (REC) programme
aimed to contribute to the further development of an area where equality and the rights of persons, as enshrined in the EU Treaty, the Charter and international human rights conventions, are promoted and protected. In parallel, the
Europe for Citizens (EfC) programme
supported initiatives to strengthen remembrance of recent European history and to enhance civic participation at EU level. Since 2021, the
Citizens, Equality, Rights and Values (CERV) programme
has sought to support and develop open, democratic, equal, inclusive and rights-based societies founded on the rule of law. This includes a vibrant and empowered civil society, encouraging democratic, civic and social participation, and cultivating the rich diversity of European society, based on common values, history and memory.
Considering their legal bases requirements and the links between the three programmes, the Commission carried out a joint evaluation with three components: (1) the ex post evaluation of the REC programme, assessing its longer-term impact and sustainability effects, based on the results of the
first part of the ex post evaluation
completed in 2022; (2) the ex post evaluation of the EfC programme, assessing the results achieved, and its long-term impact and sustainability effects; and (3) the interim evaluation of the CERV programme, assessing its preliminary achievements.
The first part of the ex post evaluation of the REC programme aimed to report to the European Parliament and the Council as provided for by the programme’s Regulation. Considering that a significant number of projects were still ongoing, the first part of the ex post evaluation assessed the preliminary achievements of the programme, which fed into the design of the activities of the CERV programme. This Staff Working Document presents the findings of the second part of the ex post evaluation; it focuses on the long-term impacts and sustainability of the effects of the REC programme.
Backward-looking, the evaluation assesses to what extent the respective activities and expected outcomes achieved these three programmes’ objectives, against the effectiveness, efficiency, coherence, EU added value, and relevance criteria. The evaluation also assesses the synergies between the three programmes and the value added of the new programme architecture. In a forward-looking perspective, the evaluation provides lessons learned for the ongoing implementation of the CERV programme as well as the following funding cycle.
In terms of scope, the ex post evaluations of the REC and EfC programmes cover the implementation from 1 January 2014 until 31 December 2020 and all activities that have taken place in all participating countries during that time. The interim evaluation of the CERV programme covers the implementation period from 1 January 2021 until 31 December 2023, and all activities taking place in all participating countries during that time, namely all EU Member States and associated non-EU countries.
1.2.Methodology framework and data limitations
This evaluation – supported by an external study (hereafter the ‘supporting study’) – combines and triangulates qualitative and quantitative data sources and methods; it is underpinned by extensive desk research. It was carried out by the European Commission’s Directorate-General for Justice and Consumers (DG JUST).
The evaluation takes stock as well of the preceding impact assessment to analyse a possible proposal for a European Culture, Rights and Values programme (2018), and acknowledges the REC interim (2018), the first part of the REC ex post (2022) and the EfC interim (2018) evaluations. A wide range of stakeholders was consulted, including EU Member States that are also members of the relevant Programme Committees, programmes’ applicants and beneficiaries, expert groups, agencies, the programmes’ contact points, civil society organisations and the public. Overall, more than 1 000 stakeholders provided feedback during the entire consultation process. Dedicated methods and tools were used to conduct the consultations: a questionnaire-based online public consultation, interviews, focus groups, deliberative workshops, and targeted surveys. They complemented data and information collected through other methods, such as the desk research and case studies.
Four main data quality challenges were identified and mitigated:
•Limited data comparability: the REC data analysis for the years 2014 and 2015 presents caveats related to challenges in retrieving data from obsolete corporate tools as well as possible different approaches to text mining in the first part of the ex post evaluation. Considerable time was dedicated to build an updated dataset and to clarify the approaches used in analysing data.
•Overemphasis on experiences with ongoing programme: interviewees who were recurrent beneficiaries from REC, EfC and CERV programmes tended to focus their feedback on CERV. The number of interviews carried out on REC and EfC was sufficient to mitigate this limitation and acquire the information needed for a comprehensive assessment of each programme.
•Limited participation by national stakeholders in deliberative workshops: despite significant efforts in reaching out, it was not possible to engage national stakeholders from four Member States as initially planned. However, findings remain robust as these Member States were represented in other consultation activities.
•Smaller number of expert interviews to validate results: a lower number of experts than initially planned responded to requests for consultation. However, findings were reviewed by a sufficient number and variety of both independent and Commission’s experts.
2.What was the expected outcome of the intervention?
This chapter is organised in three parts. Section 2.1 describes the context and rationale for the three programmes at the time of their preparation, Section 2.2 depicts the intervention logics, and Section 2.3 explains the approach to establish the points of comparison.
2.1.Description of the interventions and their objectives
Rights, Equality and Citizenship programme
The Rights, Equality and Citizenship (REC) programme was established in 2014 as a new programme merging the 2007-2013
Daphne III programme
, the
Fundamental Rights and Citizenship programme
and two of the policy areas of the
Progress programme
, namely anti-discrimination and gender equality.
The 2011 impact assessment preceding the REC programme identified the following main drivers and challenges:
·insufficient knowledge of the EU acquis and of EU policies by the persons who derive rights from it;
·insufficient knowledge of the EU acquis and of EU policies by the relevant practitioners, leading to inconsistent and insufficient application of EU law and policies;
·insufficient cooperation and exchange of information at transnational level and lack of mutual trust between authorities, such as judicial authorities;
·need for concrete information, evidence, research and good practices concerning the situation and the needs on the ground, in order to feed to the development of EU legislation and policies.
Moreover, the economic crisis had lowered the confidence that European governments were going to address issues of discrimination with the same level of funding and sense of priority. The REC programme was to address these drivers and challenges as a custom funding programme at EU level with as general objective to contribute to the further development of an area where equality and the rights of persons as enshrined in the Treaty on the European Union, in the Treaty on the Functioning of the European Union, in the Charter of Fundamental Rights and in the international human rights conventions to which the Union has acceded, were promoted, protected and effectively implemented. This general objective was to be achieved through nine specific objectives:
1.to promote the effective implementation of the principle of non-discrimination on grounds of sex, racial or ethnic origin, religion or belief, disability, age or sexual orientation, and to respect the principle of non-discrimination on the grounds provided for in Article 21 of the Charter of Fundamental Rights of the European Union (RDIS);
2.to prevent and combat racism, xenophobia, homophobia and other forms of intolerance (RRAC);
3.to promote and protect the rights of persons with disabilities (RDIB);
4.to promote equality between women and men and to advance gender mainstreaming (RGEN);
5.to prevent and combat all forms of violence against children, young people and women, as well as violence against other groups at risk, in particular groups at risk of violence in close relationships, and to protect victims of such violence (RDAP);
6.to promote and protect the rights of the child (RCHI);
7.to contribute to ensuring the highest level of protection of privacy and personal data (RDAT);
8.to promote and enhance the exercise of rights deriving from citizenship of the Union (RCIT);
9.to enable individuals in their capacity as consumers or entrepreneurs in the internal market to enforce their rights deriving from Union law, having regard to the projects funded under the consumer programme (RCON).
The programme was to finance different types of actions, such as analytical and monitoring activities, training activities, mutual learning, cooperation, awareness raising and dissemination activities, information and dissemination activities, as well as support for EU level actors and non-governmental organisations (NGOs) whose activities contribute to the implementation of the objectives of the programme. Funding was provided through action grants, operating grants and procurement contracts, of which the latter covered activities such as studies and evaluations, conferences and campaigns.
In terms of geographical coverage, all EU Member States, Iceland and Serbia were eligible to participate in the programme. Liechtenstein was eligible for the specific objectives RDIS, RRAC, RDIB and RGEN.
The REC programme was implemented by the European Commission’s Directorate-General for Justice and Consumers via direct management, with the collaboration of the Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL) as well as the Directorate-General for Communication (DG COMM), the Directorate-General Statistical Authority of the European Union (DG ESTAT), and the Directorate-General for Digital Services (DIGIT).
Europe for Citizens programme
Since the Maastricht Treaty was adopted in 1993 introducing the concept of European citizenship, the EU recognised the need to bring itself closer to citizens and to enable them to fully participate in building the EU project, while emphasising its common and shared values. The Europe for Citizens (EfC) programme was one of the instruments to achieve this political objective.
The first fully-fledged citizenship programme under the name ‘Europe for Citizens’ was implemented from 2007 to 2013, following the Community Action Programme, which awarded grants in the field of active European citizenship from 2004 to 2006. It was followed by the second iteration of the programme, which was established for the period 2014-2020, and which aimed to bring the Union closer to its citizens. The programme was to achieve this by contributing to citizens’ understanding of the Union, its history and diversity, and by fostering European citizenship and improving conditions for civic and democratic participation at Union level.
The EfC programme had two specific objectives, which translated into a two-strand approach to funding:
a)to raise awareness of remembrance, the common history and values of the Union and the Union’s aim namely, to promote peace, the values of the Union and the well-being of its peoples, by stimulating debate, reflection and the development of networks;
b)to encourage the democratic and civic participation of citizens at Union level, by developing citizens’ understanding of the Union policy making process and promoting opportunities for societal and intercultural engagement and volunteering at Union level.
The programme supported a wide range of activities and organisations promoting active European citizenship through the involvement of individual citizens. The two strands of the programme were complemented by a cross-cutting horizontal action for analysis, dissemination and use of the projects’ results. The programme had two funding mechanisms: action grants and operating grants. Action grants provided for projects within both strands. Operating grants funded European public policy research organisations (think tanks) and European civil society organisations (CSOs).
Besides the 27 EU Member States and the United Kingdom, which was a Member State until its official withdrawal from the EU on 31 January 2020, the EfC programme was implemented in six other participating partners: Albania, Bosnia and Herzegovina, Montenegro, North Macedonia, Serbia, and Kosovo*.
The programme was implemented by the European Commission via direct management. It was initially managed by DG COMM, then by DG HOME; as of 1 January 2020, the programme was transferred to DG JUST. Implementation was delegated to the Education, Audiovisual and Culture Executive Agency (EACEA) for the whole period.
Citizens, Equality, Rights and Values programme
The Citizens, Equality, Rights and Values (CERV) funding programme is the only EU programme whose main objective is to specifically seek to protect and promote the rights and values enshrined in the Treaties and the EU Charter of Fundamental Rights to sustain open, rights-based, democratic, equal and inclusive societies based on the rule of law.
Established for the period 2021-2027 as a merge of the previous REC and EfC programmes, the CERV programme received, thanks to the will of the European Parliament also supported by the Council and the Commission, a much bigger envelope (EUR 1.5 billion) than what was initially proposed (EUR 641 million). An important feature of the future CERV programme was support to CSOs allowing them to perform their activities independently. This led to the creation of the Union values strand, which aims at supporting CSOs active in the protection and promotion of EU rights and values, including at grassroots level.
The programme has four strands:
•Union values – to protect and promote Union values;
•Equality, rights and gender equality – to promote rights, non-discrimination, equality (including gender equality), and advancing gender and non-discrimination mainstreaming;
•Citizens’ engagement and participation – to promote citizens engagement and participation in the democratic life of the Union, exchanges between citizens of different Member States, and raising awareness of the common European history;
•Daphne – to fight violence, including gender-based violence and violence against children.
The specific objectives corresponding to the four strands outlined above are underpinned by the general objective as set out in the programme Regulation to ‘protect and promote rights and values as enshrined in the Treaties, the Charter and the applicable international human rights conventions, in particular by supporting CSOs and other stakeholders active at local, regional, national and transnational level, and by encouraging civic and democratic participation, in order to sustain and further develop open, rights-based, democratic, equal and inclusive societies which are based on the rule of law.’
The programme applies in all EU Member States. The Union values strand is only open to the EU Member States, while the other three strands are open to non-EU countries which are associated to the programme. By November 2024, the following non-EU partners were participating in the CERV programme: Albania, Bosnia and Herzegovina, Kosovo, Serbia and Ukraine.
The programme is implemented through grants, procurement, as well as indirect management actions implemented with the support of international organisations. Along with traditional open calls for proposals, by means of which grants are awarded for projects to beneficiary organisations, the programme also provides for financial support to third parties (or re-granting to grassroots organisations via intermediaries).
The funding is predominantly directly managed by the Commission (DG JUST) (in particular, the equality, rights and gender equality strand, and the Daphne strand), with delegation to EACEA for the implementation of the calls for proposals under the Union values strand and the citizens’ engagement and participation strand. Funding for disability under the equality, rights and gender equality strand and under the Union values strand is co-delegated to DG EMPL. Funding for the European Citizens’ Initiative is co-delegated to the Secretariat-General.
2.2.Intervention logics
The intervention logic diagram for the REC programme (Figure 1) illustrates the drivers of the programme and how the programme expected to tackle the different needs. In the short term, through the use of action grants, operating grants and procurement, the programme aimed to support analytical activities, training activities, mutual learning, cooperation, awareness raising and dissemination activities. The programme also planned to support main actors who, through their activities, contributed to the implementation of the objectives of the programme. With the help of these concrete outputs, the programme was expected to achieve the long-term results shown in the intervention logic diagram and, as a consequence, fulfil the general and specific objectives of the programme.
Figure 1 - Intervention logic of the REC programme
The intervention logic of the EfC programme (Figure 2) illustrates the inputs, outputs, outcomes and impacts which it intended to reach in order to meet the target groups’ needs. The intervention logic also displays the causal links between the drivers and challenges, objectives, input, activities, output, outcomes and impact. The needs were considered in the light of the evolving policy and socio-economic context related social challenges, such as the COVID-19 pandemic. The inputs refer to the resources committed to implement the activities and to produce the planned outputs. The programme’s Regulation envisaged four types of activities, ranging from mutual learning and cooperation activities, structural support to specific organisations, analytical activities, as well as awareness-raising and dissemination activities designed to use and further increase the value of the results of the supported initiatives and to highlight good practices.
Figure 2 - Intervention logic of the EfC programme
The intervention logic for the CERV programme (Figure 3) summarises the causal links between needs, objectives, inputs, activities, outputs, results, impacts, external factors and other EU policies. For ease of understanding, the intervention logic also includes problems, drivers and assumptions. It is based on the following underlying assumptions:
·concentrating funding in a single architecture increases efficiency;
·awareness-raising leads to appropriation of knowledge, knowledge leads to more positive perceptions, action, engagement;
·cooperation and exchange create critical mass in the dissemination of results.
As with any intervention, the impact is expected to be greater if there is coherence with other EU policies or other actors in the same space, e.g. other programmes and funds with similar objectives, and there are external factors which could not have been predicted which have to be contended with (COVID-19 and Russia’s war of aggression against Ukraine).
Figure 3 - Intervention logic of the CERV programme
2.3.Points of comparison
The ex post evaluation of the REC programme used the 2011 impact assessment and the 2015 ex post evaluation of the three predecessor programmes as points of comparison for measuring achievements. In addition, the REC Regulation set out performance indicators which served as a basis for monitoring and evaluating the achievement of the programme’s specific objectives. The first part of the ex post evaluation provided an overview of the programme’s achievements based on contracts and grant agreements signed until 31 December 2020. As, at that time, a considerable number of projects were still ongoing (more than 40%) or were still to be awarded through the 2020 calls for proposals, the current report completes the previous one, including by analysing the contracts and grant agreements signed as of 1 January 2021. Annex II.2 presents the points of comparison for both the general and specific objectives (see Table 3).
The ex post evaluation of the EfC programme assessed the EfC performance starting from the situation described in the interim evaluation carried out in 2018. The Regulation did not provide for any indicator for the general objective; hence, the evaluation used as point of comparison the percentage of EU citizens feeling European based on Eurobarometer surveys and measured throughout the programming period. As regards the two specific objectives, the evaluation referred to the performance-related indicators provided for in the Regulation. The values of these indicators were measured through programme monitoring data as well as data extracted from final reports of grant beneficiaries. Annex II.2 presents the points of comparison for both the general and specific objectives (see Table 4).
The interim evaluation of the CERV programme used as points of comparison the scenarios outlined by the 2018 impact assessment that accompanied the proposal for a Regulation establishing a Rights and Values programme. The main aim driving the proposed scenario was simplification and further exploitation of the potential of the four funding programmes to promote EU values and increase EU added value, namely the REC programme, the EfC programme, the Creative Europe programme, and the Justice programme. The scenario merged the REC, EfC, and Creative Europe programme into the ‘European Culture, Rights and Values programme’ while maintaining the Justice programme separate. Ultimately, two programmes (REC and EfC) were merged into the CERV programme, with the Creative Europe programme remaining separate. Additionally, the interim evaluation referred to the performance monitoring framework established for the CERV programme as per the requirements of the Regulation, and its set of indicators. Annex II.2 presents the points of comparison for the programme’s objectives (see Table 5).
3.How has the situation evolved over the evaluation period?
This chapter presents the progress made during the period covered by the ex post as well as the interim evaluations compared to the baselines.
3.1.Rights, Equality and Citizenship implementation
The REC programme was allocated a budget of EUR 439 473 000 for the period 2014 to 2020. The annual budget increased from EUR 54.16 m in 2014 to EUR 67.91 m in 2020. The biggest share of the programme’s budget (almost 26%) was allocated to the specific objective to prevent violence (Daphne, RDAP). The second largest budget allocation (22%) was dedicated to promoting non-discrimination (RDIS).
Figure 4 - Planned funding, shares by specific objective (2014-2020) (%)
Source: First part ex post evaluation of the REC programme (Annual Work Programmes 2014-2020)
Figure 5 presents the total of funding requested through grant applications and shows per specific objective the split between the budget requested by unsuccessful proposals and the budget granted to successful ones.
Figure 5 - Funding requested in applications and funding committed via grants by specific objective (2016-2020) (million EUR)
Source: Data on funding requested is based on the first part ex post evaluation (for 2016-2020). Data on commitments is an elaboration of data based on ABAC (August 2024). Data on 2014-2015 RDIB applications not available.
Over the implementation period of the REC programme, 81 calls for proposals were planned for which at least 4 903 applications were submitted, resulting in the award of 950 grants (745 action grants, 175 operating grants, and 30 direct grants). In addition, 492 procurement contracts were signed.
Success rates varied across specific objectives. The three specific objectives with the highest number of applications (RDAP, RDIS and RRAC) had success rates of respectively 16%, 20% and 12%. RCHI and RCIT had success rates of respectively 23% and 14%. The RDAT and RGEN specific objectives included both restricted and unrestricted calls, hence overall success rates are not available.
For procurement contracts, the highest number was dedicated to the specific objective RDIS, followed by the specific objective RDIB. The 79 unassigned contracts served the programme’s general procurement needs.
Figure 6 - Number of procurement contracts signed by specific objective (2014-2020)
Source: Elaboration of data based on ABAC (August 2024).
The REC programme geographically covered the entire European Union, with coordinators and/or beneficiaries from all Member States. While most grants awarded had a coordinator from Italy, the highest budget was committed in Belgium. The Member States with the lowest number of coordinators were Luxembourg and Denmark. In addition, few grants were also awarded in Iceland and Serbia.
Figure 7 - Distribution of grants awarded and budget committed (2014-2020), by country of coordinator (EUR million) (%)
Source: Elaboration of data based on ABAC (August 2024).
Shifting the focus from the geographical coverage to the type of activities funded by the REC programme, training (including capacity building) was clearly the predominant activity in more than half of the funded projects as of 2016, followed by analytical and monitoring activities, and awareness-raising and dissemination activities (see Figure 9). The projects marked as ‘other’ count for 6% of the total and included activities supporting the target groups and stakeholders.
Figure 8 - Number of grants awarded, by activity type (2014-2020)
Source: Interim evaluation (2014-2015, n=213), first part ex post evaluation (2016-2020, n=604), elaborations on project documents (n=21).
Analysis of a sample of 550 projects identified 207 cases of cross-border cooperation (38%), with a peak in 2017 (58 cases). In addition, in the period between 2016 and 2020, a total of 157 transnational networks were created, of which one third in 2017.
3.2.Europe for Citizens implementation
The EfC programme was initially allocated an overall budget of EUR 185.5 million EUR. The programme funded activities under the two strands – with approximately 20% of the budget allocated to the European remembrance strand and approximately 60% to the democratic engagement and civic participation strand – and a cross-cutting ‘valorisation’ activity (approximately 10% of the budget). The remaining 10% of the total budget was allocated to the programme management, covering EACEA administrative expenditure. Valorisation grants to exploit cross-cutting opportunities and learning included grants contributing to the cost of providing National Contact Points (NCPs).
The figure below presents an overview of budget implementation over the lifetime of the programme. Annual work programmes accounted for EUR 165.1 million initially planned, while EUR 175 million were committed (which accounts for the budgetary modifications) and EUR 168.4 million paid.
Figure 9 - Programme budget (planned, committed, paid) (2014-2020) (EUR million)
Legend: WP=as planned in the Annual Work Programme; CO=committed; PAY=actual payments. Source: elaboration based on monitoring data (2024).
Over the implementation period of the EfC programme, 14 222 applications were submitted, resulting in the award of 2 529 grants (2 462 action grants and 67 operating grants), indicating a success rate of ca. 17.8% across all Member States and programme strands. Success rates varied greatly, ranging from only 6% for civil society projects to 30% for proposals in the first phase of town twinning (TT).
Figure 10 - Number of submitted and selected projects by strand (2014-2020)
Source: Elaboration from Europe for Citizens 2020 Report Europe for Citizens Programme Results.
The number of grants awarded fluctuated over the years, peaking in 2019.
Figure 11 - Number of grants awarded, by year (2014-2020)
Source: Elaboration based on monitoring data and annual results reports (2024).
The total amount spent on the remembrance actions (Strand 1) was EUR 34.78 million. EUR 126.84 million were respectively spent on democratic engagement (Strand 2) and EUR 6.78 million on the cross-cutting horizontal action.
Grants were awarded in all 27 Member States. The lowest number of grants were in Luxembourg (6), and the highest in Italy (320). As a percentage of all applications selected, Italy, Hungary and Slovakia stood out, respectively with 12.7%, 12.3% and 11.1% of the total.
Figure 12 - Applications submitted and selected by EU Member State (2014-2020) (%)
Source: Elaboration from Europe for Citizens 2020 Report Europe for Citizens Programme Results.
During the lifetime of the programme, applications were received from participants in Albania, Bosnia and Herzegovina, Montenegro, North Macedonia, Serbia and Kosovo. Taken together, applications from these participating countries accounted for 4.71% of all submitted and 3.57% of all selected applications. More than half of the submitted and selected applications from non-EU countries were from Serbia. Grants were also awarded in the United Kingdom.
Overall, the programme exceeded almost all targets established for the core performance indicators, as detailed in Table 4 (Annex II.2). Over the lifetime of the programme, EfC beneficiaries estimated to had involved over 9.3 million direct participants across both strands, with 28% of the direct participants reportedly engaged in civil society projects (CIV) and 18% in remembrance projects (REM). Over half (54%) of the direct participants were reportedly involved through the operating grants (CIV-OG).
Figure 13 - Total direct participants across strands (2014-2020) (%)
Source: Elaboration based on monitoring data and final project reports (2024).
The programme achieved a balanced representation of male and female participants in activities within both its strands: 50% (CIV), 55% (CIV-OG) and 56% (REM). Almost half the estimated direct participants were aged under 30. Notably, the remembrance activities attracted the highest number of younger participants (70%) as shown below.
Figure 14 - Age of participants disaggregated by strand (2014-2020) (%)
Source: Elaboration based on monitoring data (2024).
Over the lifetime of the programme, beneficiaries of all EfC actions estimated to had reached 388 103 406 indirect participants across the EU. As shown in the figure below, the combined reach of the operating grants in the civil society and remembrance strands (CIV-OG and REM-OG) was the most significant, reportedly accounting for over three quarters of persons indirectly reached by the programme.
Figure 15 - Number of indirect participants (2014-2020)
Source: Elaboration based on monitoring data and final project reports (2024).
At programme level, 70% of projects included a transnational partnership (i.e. having at least two participating organisations from different countries). The percentage of transnational partnerships was the highest in civil society projects (CIV).
Figure 16 - Number of transnational partnerships disaggregated by strand (2014-2020)
Source: Elaboration based on monitoring data (2024).
The number of unique participating organisations, applying as coordinator, was 5 895 across TT, NT, CIV, CIV-OG and REM actions. The percentage of first-time applicants increased over the years, reaching 66% in 2019.
Figure 17 - First-time applicants per year
Source: Elaboration from Programme performance statement, DB2021.
3.3.Citizens, Equality, Rights and Values implementation
The programme is progressing well towards the milestones and targets linked to its objectives. In terms of the core performance indicators on number of people reached by project activities as set out in the programme Regulation, all milestones have not only been met but exceeded, and often by a significant amount (see Table 5, Annex II.2).
The projects awarded in 2021-2023 are expected to reach out to at least 44 million people:
·People expected to be reached by training activities: 563 215;
·People expected to be reached by mutual learning and exchange of good practices activities: 4 471 197;
·People expected to be reached by awareness-raising, information and dissemination activities: 39 352 938.
The number of CSOs reached by support and capacity building activities is also on track to meet its targets in all four strands. Between 2021 and 2023, 3 033 CSOs were reached by support and capacity building activities across all Member States. Most CSOs were reached under Strand 1 – Union values (n=1 266), with a notable increase in 2023, which reflects the number of third parties reached through the calls for intermediaries (financial support to third parties).
Figure 18 - CSOs reached by support and capacity building activities by strand and year
Source: Elaboration based on monitoring data (June 2024).
The number of transnational networks and initiatives focusing on European memory and heritage as a result of programme intervention is the only core performance indicator that shows progress below the 2024 milestone (399 compared with the milestone of 1 000). To be noted, however, that due to the cut-off date for data collection for this evaluation, the 2023 data for this indicator was incomplete.
Indicators related to the general objectives of the programme also point to positive results, except for awareness of the rule of law, where progress has remained slow (see Annex II.2). However, some of these indicators assume causality between the programme’s effects and Eurobarometer results, even though many factors are likely to play a role (not least the potential for political changes in the EU until 2027) rather than the CERV programme alone.
Indicators linked to the specific objectives of the programme show promising results in terms of metrics intended to measure changes in the behaviour, perceptions and practice of participants (who are target groups of projects) (see Annex II.2).
The data below presents the state of play of the physical implementation of the programme over the period 2021-2023.
1 093 grant agreements were signed, and EUR 335 859 932.03 were committed, of which almost half (49%) under the Union values strand.
The average size of a grant was EUR 311 877.32, while the average number of project partners in a consortium was 5.
The application success rate of the programme overall was circa 31%, highest under the citizens’ engagement and participation strand (58%), indicating that the funding allocation under this strand is most effectively meeting the needs on the ground. The Union values strand also demonstrated a relatively high success rate (48%), largely due to the specific types of actions funded, such as operating grants to strategic European networks and calls to intermediaries for re-granting to grassroot organisations. Nevertheless, while the limited number of applications due to the specificity of these calls contributes to a higher success rate, it does not imply that the funding is sufficient to fully address the needs on the ground.
The success rates under the equality, rights and gender equality strand and the Daphne strand were markedly lower, at 13% and 20%, respectively. These were also the two strands with the highest proportions of rejected proposals scoring above the 70-point threshold (71% and 54%, respectively), because these two strands cover a number of key EU policies but have, proportionally, a rather limited budget. Consequently, many excellent project applications (sometimes even with a score of 95/100) could not be funded due to the budget limitations.
Figure 19 - Number of applications and grants awarded by strand – action grants (2021-2023)
Source: Elaboration based on monitoring data (June 2024).
The largest proportion of budget committed went to CSOs (77%, EUR 258 709 334.97), which is in line with what would have been expected at mid-term in the programme.
Figure 20 - Budget committed by type of beneficiary (2021-2023)
Source: Elaboration based on monitoring data (June 2024).
The type of activities most frequently granted was mutual learning and exchange of good practice activities (n=526), followed by awareness-raising, information and dissemination activities (n=173), and training activities (n=124). There was a smaller number of analytical and monitoring activities granted (n=23) and 9 ICT tools.
In 2021-2023, altogether EUR 42 733 641 were committed for 206 procurement activities.
The type of activities most frequently contracted (procurement) was ICT tools (n=82), followed by awareness-raising, information and dissemination activities (n=58), mutual learning and exchange of good practice activities (n=32), analytical and monitoring activities (n=28) and training activities (n=6).
10 indirect management actions with international organisations were implemented for a total amount of EUR 4 425 000.
The type of activities most frequently used under indirect management actions was awareness-raising, information and dissemination activities (n=6), followed by analytical and monitoring activities (n=4). No other types of activities were implemented.
Until November 2024, 23 participating States had established a NCP.
Overall, organisations participating to projects funded by the CERV programme (n=4 232) came from 41 countries, covering all EU Member States (including the Overseas Countries and Territories of Aruba, Curaçao, and Sint Maarten) and 11 non-EU countries (Albania, Bosnia and Herzegovina, Moldova, Montenegro, Morocco, North Macedonia, Norway, Serbia, Switzerland, Ukraine, and the United Kingdom). Participants from non-eligible countries took part in projects, but did not receive EU funding.
Figure 21 – Organisations participating CERV-funded projects by country (map)
Source: Elaboration based on monitoring data (May 2024). Figures include both project coordinators (i.e. beneficiaries) and partners (i.e. participants). Aruba, Curaçao and Sint Maarten are not included in this map with <5 participants each.
4.Evaluation findings (analytical part)
This chapter is structured in three parts: Section 4.1 assesses the three programmes’ performance through their effectiveness, efficiency and coherence; Section 4.2 identifies their EU added value; and Section 4.3 analyses their relevance. The overall analytical approach is detailed in Annex II.
4.1.To what extent was the intervention successful and why?
4.1.1.Effectiveness
Rights, Equality and Citizenship programme
Evidence suggests that the REC programme has been effective in meeting its objectives even though, as noted in the first part of the ex post evaluation, the REC programme lacked a robust monitoring framework that would allow the assessment of the achievement of its specific objectives against a baseline.
The programme effectively reached out to a diverse range of organisations, including public authorities, CSOs, and research entities, ensuring that broad stakeholder groups benefited from its initiatives. These included those focused on non-discrimination, gender equality, child protection, and the prevention of racism and violence, while indirectly encompassing all individuals in the EU subject to discrimination, intolerance, or violence. While target groups were generally well-distributed across the specific objectives, certain groups were more frequently linked with specific goals, indicating a flexible and adaptive approach within the REC programme to meet diverse needs and contexts effectively.
The number of grants and procurement contracts funded, as well as the proportion of the budget committed to the different specific objectives, highlight that the REC programme contributed the most to the specific objectives RDAP and RDIS. Together, these specific objectives received more than 50% of programme funding, while the specific objectives on RCIT, RCON and RDAT only accounted for 8% of the REC funding altogether. This (uneven) distribution of funding is broadly in line with the allocations as planned under the annual work programmes. However, the evaluation results clearly showed that most projects funded under the REC programme contributed to multiple specific objectives and not only to the one for which the funding had been received. Over 70% of REC beneficiaries stated that their projects had contributed to the specific objectives RRAC and RGEN, even though these specific objectives only received respectively 13% and 8% of the funding (cf. figure below). In addition, 93% of REC beneficiaries reported that their activities had contributed to RDIS.
Figure 22 - Extent to which funded activities contributed to REC specific objectives
Source: targeted survey of beneficiaries (2024).
In line with the specific objectives of the programme, one of the main achievements of the REC programme is the number of structural coordination mechanisms set up with all stakeholders, including Roma, on the implementation of the national roma integration strategies (specific objective RDIS): this number rose from zero in 2013 to one in all 27 Member States by 2023. In addition, for RGEN, the female employment rate went up from 62.4% to 66.2% by the end of the programme, while the gender pay gap decreased to 12.3%. However, it is not possible to establish causality, also considering that multiple factors beyond the REC programme may have contributed to the progress reflected in these indicators. For RDIB, funding through procurement contributed to the organisation of the access city awards, the European day of persons with disabilities and awareness-raising workshops in Member States, while action grants contributed to the implementation of the European Disability Card in the Member States.
For RDAP, the percentage of people that consider domestic violence against women unacceptable rose substantially from 84% to 96% in 2016. Nonetheless, the existing evidence does not allow the extraction of the REC contribution on its own as again multiple factors might have contributed.
Most projects reported as key results ‘support and empowerment for target groups and stakeholder engagement’ and increased knowledge and awareness’ (cf. figure below). These results are in line with the outputs and results described in the programme’s intervention logic.
Figure 23 - Key results reported in technical reports of grants in % (n=550)
Source: First part ex post evaluation (for 2016-2020, n=211) and text mining of final reports (projects closed between 1 January 2021 and 31 December 2023, n=339).
From 2016 to 2020, a total of 350 000 persons were reached through mutual learning and exchange of good practices activities and in addition, more than 2.3 million people participated in training activities.
The REC programme has shown long-term results beyond the period 2014-2020. Some of the best examples are the numerous tools and mechanisms for cross-border cooperation and transnational networks that have been created (see Section 3.1). Thanks to REC funding, stakeholders were able to work across the EU and to build long-term relationships to promote and implement their strategies over the longer-term. Moreover, stakeholders consulted also confirmed that the types of activities funded tend to produce longer-term effects (e.g. increased knowledge and awareness following training and awareness-raising activities).
During the implementation period, the programme had to deal with external factors, with the COVID-19 pandemic had the largest impact. Even though it was necessary to adjust timelines and shift project activities to online platforms, the pandemic did not appear to impact the effectiveness of the funded projects, also thanks to the measures taken by the Commission (such as flexibility in extending the duration of grant agreements). Moreover, on the positive side, the pandemic did encourage beneficiaries to make better use of digital technologies and collaborate online. An additional external factor was the perceived hostility towards CSOs, which was a factor identified in the public consultation and led e.g. to difficulties in securing co-financing.
The programme covered all Member States, with a high demand for funding in Belgium, Italy, Spain and Greece. While the high number of applications and funding from Belgium can be explained by the high concentration of EU level CSOs based in Brussels, Italy was prominent through its significant number of applications and amount of funding requested, indicating strong awareness and mobilisation efforts.
Overall, stakeholders were satisfied with how calls were publicised and information disseminated. However, a large proportion of beneficiaries considered the REC programme not as well known in some countries as it should have been. Especially smaller, grassroots organisations faced challenges in accessing information about funding opportunities, suggesting the need for better outreach and support. Nevertheless, the evaluation results showed that the programme has been effective in addressing a diverse range of target groups, including public authorities, CSOs and research entities.
Europe for Citizens programme
The EfC programme achieved its general and specific objectives. The Programme Performance Statement referenced the percentage of EU citizens feeling European as a key indicator for the achievement of the general objective to contribute to citizens’ understanding of the Union, its history and diversity, to foster European citizenship and to improve conditions for civic and democratic participation at Union level. The target was for this percentage to remain stable at 59%. While if it is not possible to establish causality, as shown below, Eurobarometer data indicates that the percentage of EU citizens feeling European increased from 63% at the inception of the programme in 2014 to reach 72% in the spring of 2023, i.e. three years after the end of the programme.
Figure 24 - Feeling of being a citizen of the EU (% of respondents)
Source: Elaboration from Eurobarometer 2014-2023 (EB 99 p. 26 EB 97 EB 95 p. 29 p. 27 EB 92 p.14 EB 91 p.36 EB 90 p.33 EB 89 p.36 EB 88 p.39 EB 87 p. 38 EB 86 p.32 EB 85 p. 38 EB 84 p. 32 EB 83 p. 17 EB 82 p. 29 EB 81 p. 7 EB 80 p. 30 EB 79 p. 23).
Following the performance-related indicators set by Regulation 390/2014
, the programme’s beneficiaries estimated that over 9.3 million participants were directly involved in activities across both its strands, and 388 million people were involved across the EU overall (cf. figures 13 and 15). Considered in relation to the overall population in the EU in 2020, which was estimated at 447.7 million, these numbers are significant although the estimate should be made cautiously because of the caveats concerning the quality of the reporting on participant numbers and the difficulty of accounting for possible duplicates. The programme also supported the implementation of over 2 500 projects across the EU and 14 707 participating organisations across all actions. It is therefore reasonable to argue that the programme might have contributed at least to some extent to the progress recorded in the Eurobarometer’s citizenship survey over the years.
The programme exceeded the targets set for the core performance indicators, as summarised in Table 4 (Annex II.2). Beneficiaries considered that the specific objectives had been met successfully and that their projects had achieved their expected results. A third of respondents to the public consultation recognised that the programme had been successful in raising awareness of the common history and values of the EU and in increasing participation in civic and democratic life. The existence of separate strands for remembrance and civil society activities was helpful in providing distinct and separate contributions to achieving the general and specific objectives.
The EfC programme supported projects which generated results with potential for long-term impacts. Its networking value (i.e. the facilitation of cooperation on an international scale across the EU, the creation of networks in niche thematic areas and the exchange of best practices with like-minded organisations), low administrative burden, inclusivity (i.e. the opportunity for CSOs of all sizes to participate and the direct inclusion of EU citizens in activities) and broad thematic scope were success factors which enhanced long-term impact. Qualitative insights and quantitative data drawn from consultation activities pointed to the sustainability of the actions funded under the programme. Several examples of the ways in which activities continued after the end of a project or generated an impact beyond the project’s expected results were identified, e.g. the development of policy initiatives at local level to provide more assistance for NGOs working with vulnerable groups, and the creation of regional stakeholder dialogue platforms.
The most significant unexpected effects on the programme were linked to the COVID-19 pandemic, which resulted in projects facing delays or significant challenges to their implementation. For instance, there was a limit to the extent to which town-twinning projects could adapt to travel bans. However, this did not necessarily affect outcomes because beneficiaries adapted quickly and, in the case of the REC programme, the measures taken by the Commission were effective. There were beneficial side-effects from the accelerated digitalisation imposed by the pandemic, which led to some CSOs reaching out through virtual tools to more people than planned.
The programme was mainly visible to larger, well-networked eligible organisations with a strong focus on seeking funding opportunities. While beneficiaries were satisfied with the amount of information available, they highlighted ways in which they thought the programme could have been more proactively promoted, e.g. through social and interactive media. Nonetheless, the programme successfully attracted a growing number of first-time applicants over the years (cf. Figure 17), which indicates that potential applicants, including local CSOs and authorities, had been or had become aware of the programme.
Grants were awarded to beneficiaries in all Member States (cf. Figure 12). Several factors might have contributed to the distribution of funds across Member States, such as differing needs and the possibility of national funds being more widely available in some Member States than others.
Citizens, Equality, Rights and Values programme
At mid-term, the CERV programme is on track to achieving its objectives. The projects and their proposed activities and outputs are in line with the types of action envisaged by the programme across all strands. Moreover, the programme has already surpassed most of its indicator targets. Based on the evidence collected, it is likely that the outcomes envisaged in the intervention logic will be achieved, even though it is still too early to make any definitive judgements.
There is a strong indication that the selected projects align with the objectives of calls and thus contribute to programme results. Based on the evidence collected for this evaluation, awarded projects were in line with EU policy priorities, addressed a real need in the field, were based on well-established methodologies and strong partnerships established in advance, had a strong EU dimension and offered EU added value, and planned realistic and sustainable results. There was consensus across all stakeholders consulted that the re-granting mechanisms in the strands of Union values and Daphne addressed important needs in the civil society sector.
At the same time, the evidence collected also showed that the need for increased EU funding dedicated to values, rights, and citizenship, remains unaddressed. The shortage of resources allocated to these critical areas may limit the ability to fulfil the CERV objectives.
External factors, such as COVID-19, Russia’s war of aggression against Ukraine, Brexit, and specific national contexts, have had a limited effect on the programme overall. While COVID-19 led to delays in the implementation of some projects, this does not appear to have had any effect on the CERV programme as such. Moreover, COVID-19 and Russia’s war of aggression against Ukraine have been reflected in the 2021-2022 and 2023-2025 work programmes, thus maintaining the programme’s relevance.
The programme’s visibility mainly relies on the CERV NCPs and information webinars. Information is also shared via several other channels, which reach out to potential applicants throughout the EU, even where no NCP has yet been appointed, such as a DG JUST newsletter launched in 2019, the EU Funding & Tenders portal, call for proposals information sessions and others. The programme does appear to be moderately well known in EU Member States, but mainly by larger organisations with international networks or in major cities. The efforts of NCPs to increase the visibility of the programme and stakeholder engagement were widely appreciated; however, the evidence highlighted the scope for improving awareness about the CERV programme, particularly outside the strict confinements of its own target groups, to attract potentially new target groups and improve the digital and innovation aspects of the programme.
The programme covers all EU Member States and a number of non-EU countries that opted to participate in the programme on a voluntary basis. The geographical balance has also improved under the CERV programme with a higher proportion of projects in eastern Europe, than under the predecessor programmes. While geographical balance is not an objective of the programme per se, evidence collected for this evaluation denoted that a ‘fair’ representation in terms of number of beneficiaries had been accomplished in Austria, Ireland and Italy, while in terms of EU contribution, in Belgium, Finland, and Spain. Evidence showed an underrepresentation of larger EU Member States mainly in western Europe and the Nordic countries (apart from Finland as regards EU contribution), while in 16 EU Member States there has been an over-representation, most pronounced in Cyprus and Malta.
Considering these findings, it does not appear that the absence of NCPs in Belgium, Bulgaria, Ireland, Malta, Hungary and Poland has had an impact on geographical balance during the 2021-2023 period, taking into account the presence of other information channels that target potential applicants. Nonetheless, Poland is the only country that showed an underrepresentation, and it cannot be known what impact having a NCP might have had given all the other factors likely to be at work in influencing the number of applicants and beneficiaries. Overall, NCPs and the services that they provide to potential applicants are a key strength of the programme.
Moreover, evidence collected did not identify any imbalances as regards target groups.
The CERV programme demonstrates a strong commitment to mainstreaming gender equality and promoting a more inclusive and equitable approach to its activities. As mandated by the Regulation, the programme promotes gender equality and non-discrimination in all activities. To ensure effective implementation, the programme has integrated gender equality into its calls for proposals and evaluation methodology. Applicants are required to demonstrate how they will respect and monitor gender equality during project implementation, and contracted experts are trained on the gender equality tracking methodology.
This approach has contributed to capacity building among experts and improved data collection, including sex-disaggregated data. The information sessions and a publicly available Q&A document provide guidance on incorporating a gender perspective into proposals. Furthermore, each project financed through grants is also assessed and scored based on its contribution to gender equality.
Nevertheless, despite the significant training efforts undertaken to support applicants and beneficiaries, as well as to provide guidance on integrating a gender perspective, a significant number of beneficiaries continued to find including the gender perspective a challenge, oftentimes due to misunderstanding of this requirement as a ‘tick-box’ or ‘academic’ exercise. Comparisons between the gender scores of the CERV programme and equivalent EU funding programmes shows that the CERV programme, of the total programme funding available, has the largest proportion of funding dedicated to projects that specifically aim to promote and advance gender equality, which is in line with the programme’s objectives. Overall, between 2021-2023, every fourth euro of financing from grants under CERV contributed strongly to gender equality; additionally, about half of all the grants funded projects closely intertwined with the promotion of gender equality.
4.1.2.Efficiency
Rights, Equality and Citizenship programme
The REC Regulation set a budget of EUR 439 476 000 for the period 2014-2020, with yearly amounts set out in the work programmes. By August 2024, 91% of the budget committed had been spent on grants and procurement. With outstanding payments of several grants that ended in 2024, this percentage will likely further increase.
Despite the COVID-19 pandemic, most beneficiaries confirmed that they spent the funding they were awarded as planned. Nevertheless, the pandemic was still the most frequently cited factor impacting expenditures. On the one hand, it reduced of the travel and subsistence budget, while on the other hand it sometimes increased the internal costs of projects e.g. due to the (no cost) project extensions awarded or to unplanned costs linked to the shift to online seminars and podcasts.
Most beneficiaries were satisfied with the timeliness of information on calls, the eligibility criteria, the selection process and the received feedback. The introduction of the eGrants portal in 2016 led to mixed reactions: while some appreciated the ease of use, others faced technical difficulties and missed a direct contact person to clarify and solve issues.
The current evaluation confirmed the results of the first ex post evaluation that small organisations, such as civil society and grassroots organisations, while being in an ideal position to provide very relevant field expertise, struggled with securing the required 20% level of co-financing.
Stakeholders considered the application time-consuming and both the application and reporting burdensome. Some of the feedback received concerned unnecessary repetition in reporting and changes to the reporting requirements during the implementation period. A few interviewees proposed a shift towards a more results-oriented approach, which has been tackled in the meantime under the CERV programme with the lump sum funding. In addition, the number of yearly deliverables, deemed as excessively high by beneficiaries, has also been reduced under the CERV.
Despite the highlighted administrative burden of the application and reporting process, the benefits of the REC programme clearly outweighed the costs to beneficiaries. Not only is there an interest in continuing to apply for funding, but for many recurrent applicants, the programme filled a real gap, providing independent funding, and in many cases no other funding could have covered the themes and types of activities funded.
Europe for Citizens programme
Regulation 390/2014 set the budget of the EfC programme at EUR 185 468 000. The budget was increased twice during the implementation period: in 2014 by EUR 2.25 million; and, in 2020 by EUR 6.9 million. Except for 2014 and 2020, programme expenditure remained stable over the years and in line with the plans made in the respective work programmes (cf. Figure 9).
Success rates varied greatly, with a high number of good quality proposals that could not be funded (cf. Figure 10). The benefits of the EfC programme outweighed the costs to beneficiaries, with most beneficiary organisations successfully absorbing the awarded funding; those that experienced difficulties absorbing the funds did so because of the extraordinary circumstances that arose during the COVID-19 pandemic. The qualitative feedback from beneficiaries did not identify any significant barriers to applying for funding relating to language, capacity, relevance, awareness of the programme or other factors. Nearly all the EfC beneficiaries consulted considered the application process to be clear, accessible, simple, and straightforward. It is important to note however that the programme was already using lump sum funding when it was not relying on the eGrants system (as the REC programme did from 2016 onwards). The transition to the eGrants system may have presented challenges for some EfC beneficiaries, leading them to perceive that the previous procedures were simpler in comparison.
The simplifications which had taken place over the successive iterations of the programme improved accessibility to CSOs of all types and sizes. The programme was sufficiently known to attract new applicants every year, as also indicated by the increase in the number of first-time applicants (cf. Figure 17). With regard to financial aspects, the use of lump sums and unit costs was a successful feature of the programme.
Although it was challenging for some smaller or grassroots CSOs, the co-financing requirement was accepted as necessary and conducive to the leveraging of additional funds. Most beneficiaries did not face any specific difficulties in raising the additional funds from local authorities, foundations and banks depending on the project. The advance payment increases from 40% to 50% in 2019 and to 60% in 2020 made it even easier for smaller organisations to access the programme.
Consultation activities gathered limited feedback to suggest that the programme could have been implemented in a more efficient way. The programme’s funding generated worthy results, which contributed to its objectives and the wider EU policy goals and priorities. The evaluation did, however, identify a few instances in which the administrative burden could have been reduced even further, e.g. through more guidance.
Communication between the Commission and beneficiaries was open and clear. The selection process was considered generally fair and transparent, and it was noted that the Commission provided appropriate feedback on the outcomes. In general, the reporting requirements were considered to have been clear, straightforward and simple even though less experienced beneficiary organisations would have appreciated more substantial guidance and training on reporting requirements and forms. The Civil Dialogue Group, a specific feature of the EfC programme, provided meaningful inputs.
Citizens, Equality, Rights and Values programme
The analysis of the efficiency criterion confirmed that, to date, the benefits of the CERV programme outweigh the costs, which beneficiaries bear due to participation in the programme.
The main benefits of the funding were directly project-related (i.e. the ability to implement the desired approach), but also in terms of increased networking and collaboration with partners in other countries, and enhanced visibility and recognition of the organisation because it received EU funding. Based on the evidence collected through the supporting study, the opportunity to test innovative approaches to address societal challenges, involving new types of stakeholders, and benefiting their target groups through greater capacity and recognition of advocacy actions were also found to be important benefits of the funding.
However, the ‘cost-of-living’ crisis meant for many beneficiaries that the benefits of the funding were significantly reduced, with many struggling to maintain previously planned project budgets and, in some cases, having to adjust the range of planned activities.
The main effects of the new implementation features introduced by the Commission under the CERV programme have been improved predictability, reduced administrative burden, strengthened programme monitoring, and included more grassroots and smaller organisations through re-granting mechanisms.
Figure 25 - Impact of innovation on the monitoring burden
Source: targeted survey of beneficiaries (2024).
One exception to this overall simplification and burden reduction are unit costs. Their introduction at Commission corporate level appears to have added complexity, especially as they do not reflect the actual costs of activities and raise challenging limitations on travel costs.
The application process of the programme is timely, with information about the calls reaching applicants when they need it. For project beneficiaries, the main costs of the programme have been associated with the staff time and resources required to invest in the application process, with large variability between smaller organisations, who perceived these costs to be a lot higher, and larger organisations with professionalised bid teams, for whom these costs appeared to pose less of a challenge. There administrative burden has been identified as a risk for very small projects (such as town twinning), including due to the challenges of using the eGrants system, which can deter from applying when the effort required to apply is higher than the relatively limited grant amount expected.
While satisfaction with the feedback received from the Commission on applications was high, there appears to be room for improving consistency in the level of detail provided, and for making clearer what the Commission expects from applications. For example, some respondents suggested to improve clarity on terminology and methodology used as well as to introduce more specific examples and best practices to follow.
On the approach to reporting, notable improvements were made in the clarity of the reporting requirements compared with the REC programme. For example, reducing the reporting requirements has enabled projects to be more result oriented.
The limited inefficiencies found related to reporting requirements for operating grant beneficiaries. The timing of reports did not always align with their financial year, leading to requests for extensions. This was often due to the fact that the required data were not yet available at the scheduled reporting time, as it was dependent on the beneficiary’s financial year-end closure. The reporting template was designed for project-based reporting, which may not be considered suitable by all beneficiaries.
The process of requesting budgetary amendments was also highlighted as burdensome. Data collection through the EU Survey on Justice, Rights and Values - one of the programme’s monitoring tools specifically developed to collect data and feedback from participants at events organised by CERV-funded projects - was also criticised for its lack of relevance for target groups, its length, and the requirement to collect equality data, which some beneficiaries found problematic and ‘impossible’ to collect in the context of dissemination activities.
Additionally, the evaluation identified the eGrants system’s lack of user-friendliness and technical issues as an important factor impacting the efficiency of the programme. Although the system was revised recently, its effectiveness could not be assessed within the scope of this evaluation.
Finally, the small number of ICT tools contracted and granted, compared with other types of activities funded indicates that there is scope to make even better use of digitalisation opportunities at project level.
4.1.3.Coherence
Rights, Equality and Citizenship programme
The evaluation confirmed the findings from the first part of the ex post evaluation that the objectives and interventions of the REC programme were coherent with wider EU policies and priorities. This was ensured through the nature of REC programming where the priorities of the call for proposals were led by specific strategic developments and Commission’s priorities. The programme contributed to the EU justice agenda for 2020 by funding activities that focused on prevention, protection and victim support, including awareness-raising among the general public, training of professionals and capacity building of organisations and structures working on these issues.
The programme contributed to the LGBTIQ+ equality strategies by funding activities to empower LGBTIQ+ people, support victims of hate crime and promote their rights. In support of the 2010-2020 European disability strategy, activities were funded to support people with disabilities and promote their rights. The programme also funded activities to support Roma people and promote their inclusion and rights, thereby contributing to the EU framework for national Roma integration strategies.
The programme contributed to the gender equality strategy 2020-2025 and the 2016-2019 strategic engagement for gender equality by funding activities to improve equal economic independence, reconciliation of professional and private life, reducing the gender pay gap but also by funding activities aimed at ending gender-based violence. The contribution of the REC programme to promote equality between women and men and to advance gender mainstreaming was not only addressed through the specific objective RGEN, but also through calls under the specific objectives RCIT, RDIS and RDAP. In addition, gender mainstreaming and the promotion of gender equality were integrated in the programme design: gender mainstreaming and the promotion of gender equality needed to be integrated and evaluated in all funded projects, ensuring that the gender equality perspective could be part of the design, implementation, monitoring and evaluation of the activities.
The REC programme mainly contributed to the new Commission priorities established by President von der Leyen for the period 2019-2024 ‘Promoting our European way of life’ and ‘A new push for European democracy’, while to a lesser extent also to the ‘European Green Deal’, ‘a Europe fit for the Digital Age’ and ‘A stronger Europe in the world’ priorities.
The REC programme complemented other EU funding instruments, of which the following were the most frequently used by beneficiaries: Erasmus+, the Justice programme, Horizon 2020, Asylum, Migration and Integration Fund (AMIF), ESF, Creative Europe and the European Regional Development Fund. Coherence was ensured through the different stages of the programme: from the objectives set by the Regulation, through implementation (annual work programmes), and until the project kick-off. 77% of beneficiaries that received REC funding were positive that the programme was well aligned with other EU funding programmes.
About a third of REC beneficiaries consulted for this evaluation declared that they had received funding from other sources, e.g. from United Nations’ agencies or the Visegrad Fund. Beneficiaries did not report any lack of coherence or overlaps with these funding sources or with national, regional or local funds.
Europe for Citizens programme
The EfC programme was coherent with EU policies and priorities. The evaluation found that the programme was aligned with the Commission’s priorities for 2014-2020, but it also demonstrated flexibility to priorities of the new Commission as established by President von der Leyen for the period 2019-2024. Projects funded under the programme addressed critical issues such as fake news, media literacy, and e-democracy, thus aligning with EU priorities. Many addressed solidarity and social inclusion, the use of digital tools to enhance democratic engagement, the upholding of EU values; overall, all projects sought to encourage a stronger democratic participation. During the COVID-19 pandemic, increased digitalisation further underscored this alignment. The annual work programmes consistently recommended to potential beneficiaries the proactive use of social media with the aim of reaching out especially to younger generations, which paid off, as indicated by the high number of participants aged under 30 in the programme overall (cf. Figure 15). The programme facilitated bottom-up activities contributing to key EU priorities. The priorities it addressed included climate change, the European Green Deal, fundamental rights, gender equality, promoting active citizenship and understanding of EU values. The programme supported the European Citizens’ Initiative (ECI), a unique transnational instrument enabling EU citizens to engage directly in the legislative process, in line with the EU goal of enhancing participatory democracy. The programme complemented other EU programmes such as Erasmus+, Creative Europe, and the European Social Fund, supporting unique activities that enhanced its distinct contribution to EU goals. Synergies between these programmes were exploited on the ground, notably through the occasional, informal cooperation of national structures such as EfC NCPs, the Creative Europe Desks and Erasmus+ National Agencies. Feedback from beneficiaries showed that, despite its smaller budget, the programme’s features effectively promoted networking and created new opportunities for synergies. This was especially valuable for enhancing cross-border cooperation, facilitating educational and cultural exchanges through other programmes such as Erasmus+, and advancing bilateral partnerships with twinned organisations. The focus on cultural heritage and remembrance activities aligned with initiatives such as the European Year of Cultural Heritage (2018), thus highlighting its role in promoting a shared European identity. These efforts collectively contributed to the EU overarching goals of fostering democratic participation, social cohesion, and informed citizenship.
Citizens, Equality, Rights and Values programme
The CERV programme occupies a unique niche within the EU policy and funding landscape, addressing gaps where other funding is not available, as confirmed by beneficiaries and the public consultation. The CERV programme offers advantages that cannot all be found in other programmes, such as pan-EU coverage, a comprehensive range of themes, the possibility to secure independent sources of financing, a specific focus on EU values, grants of a significant size, operating grants and re-granting mechanisms. What emerges clearly overall is that the CERV programme occupies what would otherwise be a largely empty space in the values and rights funding landscape.
Nevertheless, as the funding gap is significant, beneficiaries, if they can, also use other sources of funding, such as the Visegrad Fund. For some specific topics, beneficiaries seek to complement funding under the CERV programme with funding from other EU programmes, such as Erasmus+ and Horizon Europe or with governmental funding (at national, regional and local level).
In terms of internal coherence across strands, no specific issues have arisen. Beneficiaries appear comfortable with the single architecture that has replaced the predecessor programmes and do not find the division of themes across the strands to be an issue. The new programme architecture of the CERV programme – bringing EU funding in the areas of values, rights and citizenship under one single programme – was considered to foster coherence and synergies between different policy priorities and was found to have contributed to a more holistic approach to addressing societal challenges and promoting EU values.
Programme implementation is coherent with the EU’s international commitments and objectives, including the UN Sustainable Development Goals. International commitments are systematically cross-referenced in calls. Results in EU budget performance measurement show only limited green budgeting on horizontal green priorities so far – which is also due to the nature of the programme – but there are nevertheless several projects with a climate change dimension.
4.2.How did the EU intervention make a difference and to whom?
Rights, Equality and Citizenship programme
The evidence confirmed that the results of the REC programme could only have been achieved by action at EU level.
The programme contributed to the consistent and coherent implementation of EU law and wide public awareness about the rights deriving from it. As already detailed above, increased knowledge and awareness was one of the main results reported by beneficiaries in their final reports (cf. Figure 23).
In addition, the REC programme contributed to developing mutual trust among Member States and improving cross-border cooperation, as clearly exemplified by the 157 transnational networks created between 2016 and 2020. The transnational nature of the programme allowed organisations to apply with consortia composed of organisations from different Member States, but also to implement activities with participants coming from different Member States. This was already highlighted in the first part of the ex post evaluation as one of the results that could not have been achieved exclusively through national funds.
The programme also helped to prepare and disseminate best practice and to create minimum standards, practical tools and solutions that addressed cross-border or EU-wide challenges. A large proportion of beneficiaries reported the exchange and transfer of good practices and mutual learning, as well as the development of research tools and analyses as a result in their final reports. The outputs of these activities continued to be used after the project had ended.
By providing independent funding, the programme enabled sensitive topics to be addressed that would not have been funded at national level. Nearly all beneficiaries interviewed confirmed that if the programme had not been sustained, national governments would have been unlikely to fill the gap. The programme increased the capacity of a wide range of key civil society players and human rights organisations. Several operating grant beneficiaries (e.g. EU networks) highlighted that REC funding had been fundamental to their organisation, and that they would not have been able to remain operational without REC.
Europe for Citizens programme
It is likely that the impacts identified through the EfC programme would not have been achieved without the programme’s specific EU-level support. Without the EfC programme, many beneficiaries would have struggled to implement projects on a similar scale or with the same impact.
The interim evaluation already found that the programme demonstrated clear added value at the EU level both in the aggregate effect of its impact on participants and its complementarity with other EU funding instruments and policy initiatives. Importantly, there was very strong evidence that the programme was overall unique given that funding at national or regional level to achieve the same or similar objectives were at best very limited. These conclusions were confirmed by the evidence generated for this evaluation.
The programme particularly responded to the CSOs challenges related to a lack of funding for work areas critical to European citizenship. It was crucial for activities requiring cross-border cooperation and a broader European perspective, as no similar national or regional schemes could match the programme’s scope and scale. The programme funded activities that involved citizens and organisations from multiple participating countries, fostering a sense of European identity and belonging. It therefore continued to fill what would otherwise have been a gap and created a level playing field enabling an equal access to the programme to all, except some smaller organisations.
These outcomes were quantifiable through the number of projects spanning multiple countries and the scale of cross-border collaboration achieved. The programme’s ability to bridge national differences and promote a cohesive European approach was instrumental in addressing the varied challenges faced at the national level. At programme level, 70% of projects included a transnational partnership (cf. Figure 16).
By facilitating cross-border partnerships and the exchange of best practices, the programme enabled CSOs of all sizes to engage in more inclusive and effective activities. This support was crucial in enhancing the capacity of organisations to operate on a larger scale or beyond their usual mandate. While most beneficiaries did not always consider the EfC funding to be critical, it was nevertheless instrumental in helping them achieve more than they could have done without it. The programme enabled organisations to carry out specific projects and reach broader audiences, particularly in areas where national or regional funding options have been limited. In some cases, the funding was essential for the survival of organisations operating in challenging funding landscapes.
The programme particularly emphasised transnational and cross-border cooperation, with significant focus on creating networks in niche thematic areas and promoting direct inclusion of EU citizens in activities. This approach addressed the varied challenges across Member States by enhancing participation and dialogue at the EU level, thereby contributing to a more balanced and inclusive civic space.
Citizens, Equality, Rights and Values programme
In the absence in many EU Member States of public funds in the thematic areas covered by the CERV programme, CERV funding remains a major contributor to civil society work in these areas. The programme adds value over and above that created by Member States acting alone.
Figure 26 - Importance of action at EU level
Source: Elaboration from public consultation (2024).
Most stakeholders consulted considered the funding of transnational projects addressing rights and societal challenges in Member States to be minimal to non-existent. Moreover, political challenges were found to be an obstacle for many CSOs to obtain national funds in the area of equality and non-discrimination, as well as combating violence against women. Based on the evidence collected through the supporting study, this is mainly due to national governments’ general lack of interest in these areas, as well as to political decisions or a national focus not interested in a cross-border dimension driving the selection of beneficiaries, including for EU funds under shared management. In this context, the opportunity to obtain direct funding from the Commission was underlined as a major contributor to fairness and independence.
The evidence gathered suggests that in the absence of the CERV programme a negative effect would likely be observed, possibly impacting the already shrinking civic space throughout the EU and significantly slowing down progress towards gender equality and equality overall. Feedback from stakeholders and experts further confirmed that a discontinuation of CERV funding would contribute to a further decline in the sense of European citizenship and identity, more barriers for persons with disabilities, an increase in violence against children, a rise in extremism and radicalism, and further divisions.
4.3.Is the intervention still relevant?
Rights, Equality and Citizenship programme
The REC programme remained relevant by effectively addressing both existing and emerging needs within the EU, including persistent societal challenges, such as discrimination against LGBTIQ+ people, Roma, persons with disabilities, and older people. Issues such as hate crimes, hate speech, and violence against women and children, which have arisen in specific contexts such as the online environment and the COVID-19 pandemic, were also at the core of the programme.
The pace of progress to achieve full equality and fundamental rights across the EU emphasise the continued relevance of the programme’s objectives, while the high application rates under various specific objectives confirmed the strong demand for support. The flexible programming, through the annual work programmes, ensured that the REC programme could respond to the dynamic socio-political landscape within the EU and remained relevant to the evolving needs of stakeholders and citizens.
The COVID-19 pandemic required the programme to show flexibility as regards procedures, timelines and activities. Beneficiaries were very positive about how the related challenges were handled and the fact that project progress was ensured. Beneficiaries felt that the programme was also responsive to political factors and, to a lesser extent, economic factors. As for challenges, inflation and limitations in eligibility criteria, particularly concerning non-EU countries, were noted as significant issues.
The REC programme effectively targeted marginalised and vulnerable groups, such as victims of violence, ethnic minorities, and children, in line with the critical challenges facing EU society. The programme demonstrated considerable relevance to its beneficiaries, as the funding matched their strategic goals, avoiding the need for disproportionate adjustments to their activities. The focus on capacity building, knowledge sharing, and structural support effectively addressed their requirements. As an example, many beneficiaries of operating grants noted that the capacity-building efforts supported their organisational development and long-term stability. Also, many beneficiaries highlighted that the funding enabled them to undertake projects they would otherwise not have been able to execute. This funding often served as a foundation for further project development and methodological advancements. Finally, the programme facilitated the growth and consolidation of networks, crucial for enhancing the impact of activities and fostering international collaboration.
Some areas of improvement suggested by beneficiaries were more attention to intersectionality in addressing gender and social inclusion issues; greater focus on emerging issues such as disinformation, mental health and early childhood development, as well as inclusion of non-EU countries to enhance international cooperation.
Europe for Citizens programme
The alignment with broader EU policy goals and priorities, and the fostering of a sense of European identity that were achieved by the EfC programme remain relevant objectives. By supporting the ECI, the programme enhanced democratic life, enabling citizens to influence EU policies. The Commission’s efforts to raise awareness and improve ECI accessibility demonstrated a practical approach to citizen empowerment. The programme also encouraged civic participation through grassroots initiatives, fostering a sense of belonging and mutual understanding among Europeans and providing platforms for meaningful dialogue and action on common issues. Town-twinning activities promoted cross-border interactions and cultural exchanges, fostering European identity and ongoing dialogue on integration and shared values, while projects on European remembrance contributed to a shared understanding of European history.
The EfC programme was relevant to its final beneficiaries, and mostly addressed the needs and target groups in thematic areas that remain relevant today. The programme’s structure supported a broad range of initiatives in line with the needs and priorities of CSOs, allowing them flexibility to tailor projects to specific thematic areas.
The beneficiaries’ primary focus on citizen engagement and participation highlighted the relevance of the programme, given its focus on citizens. CSOs were also a key target, reflecting the programme’s commitment to fostering open dialogue and collaboration, which is crucial for achieving EU policy goals. Public authorities were targeted for advocacy, enhancing the programme’s relevance by promoting effective dialogue between civil society and decision-makers. The inclusion of vulnerable groups also demonstrated the programme’s dedication to addressing inequalities and incorporating marginalised voices into the democratic process.
Figure 27 - ‘Who is your target audience?’ (%)
Source: targeted survey of beneficiaries (2024).
There was a consensus among beneficiaries that the programme’s priorities had aligned well with the greatest needs in Member States relating to civic engagement and remembrance. A significant proportion of respondents to the public consultation also considered that the programme’s objectives were still relevant to the current challenges and needs in their Member States. The continuation of projects funded by the programme further highlighted its relevance, as a significant proportion of recipients received funding for projects that built on earlier initiatives and previous results. This indicates that the programme’s support was in line with past needs and priorities, ensuring that funding remained relevant over time.
Citizens, Equality, Rights and Values programme
The CERV programme remains relevant, given its objectives as set out in Regulation 2021/692
. The evidence gathered for this evaluation confirms the gaps identified in the impact assessment. The EU is facing significant challenges related to equality, rights, and democracy. Despite progress in some areas, many EU citizens still experience discrimination, violence, and inequality. The COVID-19 pandemic and Russia’s war of aggression against Ukraine have amplified these issues, particularly affecting vulnerable groups such as children, people with disabilities, and women (cf. Annex VI). Overall, these challenges highlight the need for increased efforts to promote equality, rights, and democracy in the EU, including greater support for civil society and awareness-raising initiatives.
The evidence points to a very strong alignment between the needs of the target stakeholders, including of CSOs, and the actions funded under the programme, thus contributing to the EU’s capacity to respond to the above-mentioned challenges. This not only responds directly to beneficiary organisations’ missions and strategies, but also it allows them to reach new target groups. The EU-wide dimension was also considered to respond to CSOs’ ambitions and needs to pursue projects involving multiple countries to increase learning and impact.
Figure 28 - To what extent does funding under the programme align with the core needs of your organisation? (%)
Source: targeted survey of beneficiaries (2024).
Beneficiaries emphasised that CERV funding (as well as funding under the predecessor programmes) had been instrumental in addressing key priorities and initiatives that were crucial for the growth and sustainability of their organisation. The funding enabled them to implement projects that they would not have been able to implement otherwise, and these projects often created a basis (in terms of approaches and materials) for further project work.
The thematic focus of the calls published so far, has been relevant to most stakeholders consulted in this evaluation; it is one of the key strengths of the programme. These included, for example, such topics as capacity building and awareness on the EU Charter of Fundamental Rights; fighting against any form of intolerance, racism, xenophobia, discrimination; equal participation and representation of women and men in political and economic decision-making and tackling gender stereotypes; promoting democratic participation through debating the future of Europe and many others.
However, some gaps remain. These mainly focus on the greater inclusion of themes related to climate change and energy, intersectionality, service provision, disinformation, and war crimes and victim support.
The programme was also relevant to its final beneficiaries, ‘citizens’ being the most frequently identified group among those who benefited from it.
Figure 29 - Who is your target audience?
Source: targeted survey of beneficiaries (2024).
The programme has already demonstrated its flexibility in adapting to changing needs, both in terms of processes and procedures to respond to the COVID-19 public health restrictions, to the consequences of Russia’s war of aggression against Ukraine as well as thematically.
The CERV programme responds to Article 2 of the Treaty of the European Union, which states that ‘the Union is founded on the values of respect for human dignity, freedom democracy, equality, the rule of law and the respect for human rights, including the rights of persons belonging to minorities. These values are common to the Member States in a society where pluralism, non-discrimination, tolerance, justice, solidarity and equality between women and men prevail.’ It addresses needs relevant to the EU priority: ‘A new push for European democracy’, and supports a wide range of policy initiatives relevant to several of its specific objectives.
There remains scope to further capitalise on opportunities brought about by digitalisation. The programme’s approach evolved to correspond to the pace of digitalisation, which is included in the broad definitions for funding provision. However, these aspects do not appear to have yet generated interest or capacity to respond to the opportunities presented, as only a small number of ICT tools were funded thus far.
5.What are the conclusions and lessons learned?
This chapter is organised in two parts. Following a brief overview focusing on the joint assessment of the three programmes, conclusions and lessons learned are presented by programme and structured according to the evaluation criteria of effectiveness, efficiency, coherence, EU added value and relevance; they refer to findings presented under the previous chapter. Conclusions are robust, as they are based on findings demonstrated by multiple sources and objectively verifiable evidence.
Overall, the EfC and REC programmes achieved their objectives while the CERV programme is also making good progress towards its objectives. The objectives of the CERV programme remain highly relevant. As of now, there is no reason not to carry forward the general and specific objectives of the CERV strands or to change the objectives, the approach to the strands or the redistribution of themes across strands. The new features of the CERV programme have produced efficiency gains for the Commission and beneficiaries. These features have already reflected lessons learned from the EfC and REC programmes about the need for simplification and introduced changes being implemented across EU programmes. The application and reporting process under the CERV programme compares favourably with the REC programme but has added complexity for smaller organisations, such as towns and municipalities compared to the EfC programme (also due to the introduction of the eGrants system). Re-granting mechanisms have led to the CERV programme being more inclusive than the REC programme by allowing the programme to reach smaller and grassroots organisations. The EfC, REC and CERV programmes have all been coherent and complementary in their objectives with other EU funding programmes. The three programmes all filled roles in the funding landscape that Member States would in general not have had the capacity to fulfil. The CERV programme, like its predecessors, offers clear added value. It is for many CSOs, their only possible source of funding in this area as was the case under REC and the EfC. Based on the evidence collected through the supporting study, there is little likelihood of Member States to step in in the absence of the CERV programme; direct management is crucial for ensuring transparency, fairness and independence. There is a clear need for the CERV programme to continue because the challenges these programmes were set up to address persist and, in some cases, have got worst, e.g. the increasing polarisation of society, the rise in populism and extremism, and the threat to EU values. The needs that the EfC, REC and CERV programmes were set up to address are therefore still very present and seem likely to persist over the remainder of this multiannual financial framework and beyond.
5.1.Rights, Equality and Citizenship programme
Effectiveness
The REC programme achieved its general and specific objectives. The nine specific objectives were funded in line with plans established in the annual work programmes and the ones that received the highest amount of funding were also the ones where beneficiaries confirmed that their activities had most contributed. The programme covered all Member States, with a higher concentration of funding in a number of (larger) Member States (e.g. IT, ES, BE). The programme effectively addressed a diverse range of target groups, ensuring that varied populations benefited from its initiatives. Key results were achieved in the area of support and empowerment of target groups and stakeholder engagement, followed by increased knowledge and engagement of target groups. The programme reached a high number of persons, of which more than 2.3 million persons through training activities alone. The REC programme had long-lasting effects, such as the creation of long-term relations between relevant stakeholders, increased awareness among target groups about the support available and increased internal capacity within organisations that received REC funding. In addition, the project outputs of the REC-funded projects were long-lasting and used beyond the end of the funding e.g. training materials continued to be used, and online platforms established during REC projects continued to operate.
The lack of a robust monitoring framework to allow a more systematic and efficient assessment of the effectiveness of the programme was addressed under CERV through e.g. the streamlining of indicators and the introduction of dedicated data collection tools.
Efficiency
The benefits of the REC programme outweighed the costs to beneficiaries, although many beneficiaries highlighted that the administrative burden of the application and reporting process had been high. Funding distributed through the programme was largely spent as planned. Funding recipients absorbed 91% of the budget committed to grants and procurement; this percentage is likely to further increase when all projects are closed. The most cited external factor impacting on costs was the COVID-19 pandemic, although both leading to higher costs (longer project duration leading to more personnel costs) and cost savings (events being organised online, reducing travel, subsistence and organisational costs).
Building on initial lessons learned under REC, the CERV programme introduced lump sum funding to further reduce the administrative burden as well as re-granting mechanisms to increase funding’s accessibility for smaller organisations.
Coherence
There wase a high degree of coherence between the objectives and interventions of the REC programme and EU policies and priorities, as well as other EU funding instruments. The priorities of the call for proposals were led by the strategic developments and the EC priorities. The programme contributed to several major EU strategies, such as the EU justice agenda for 2020, by funding activities focused on prevention, protection and support of victims including awareness-raising amongst the general public, training of professionals and capacity building of organisations and structures working on these issues. The programme also successfully contributed to the EU framework for national Roma integration strategies by funding activities to support Roma people and promote their inclusion and rights. Finally, the programme not only addressed gender mainstreaming and promotion of gender equality in one of its specific objectives, but it also integrated them in the programme design.
The evaluation found no evidence of overlaps in the funding of activities despite some similarities noted between the REC programme and other EU funding instruments (e.g. similar objectives, or similar types of activities that can be funded). Beneficiaries reported synergies with projects funded by other EU sources, e.g. under Erasmus+, Horizon 2020, ESF, AMIF.
EU added value
The REC programme offered clear EU added value as it promoted different rights derived from Union law and their related policies, often filling gaps existing at national level. It was inherently transnational due to the way it was set up. The programme’s specific objectives all aimed to promote different rights derived from Union law. Moreover, the programme allowed organisations to apply from different Member States as a consortium and allowed for the implementation of activities with a transnational nature. As a result, the REC programme generated transnational results, notably the increased awareness and understanding of the rights and policies covered by the programme, creating or strengthening cross-border cooperation and partnerships, as well as creating best practices and tools that were used across different Member States.
The results of the REC programme could only have been achieved by action at EU level and it would have been unlikely that national governments filled the gap if REC funding had not existed in their Member State. REC funding was fundamental for many of the beneficiary organisations, who would not have otherwise been able to remain operational in the same way. This is in line with the findings of the first part ex post evaluation that identified REC funding as an important source of financing for the beneficiaries, whether public or private, to implement pilot projects and to support organisations working with target groups that would not have been otherwise available through national resources.
The learning from the REC programme justified the continuation of funding actions with high EU added value in the area of rights, equality and citizenship, namely activities that aim to create or strengthen cross-border cooperation (such as establishing transnational networks), activities that focus on mutual learning or setting up best practices or tools for EU-wide challenges, as well as activities that aim to strengthen the coherent application of EU law or that aim to increase the public awareness of their rights.
Relevance
The evaluation confirmed the findings from the first part ex post evaluation that the REC programme continued to be relevant to the (changing) needs of stakeholders and citizens in the EU. The programme addressed persistent societal challenges, such as discrimination against LGBTIQ+ people, Roma, persons with disabilities, and elderly individuals. Clear examples of how the REC programme made a difference are the European Disability Card and the creation of national Roma contact points. Issues like hate crimes, hate speech, and violence against women and children, which had risen in certain contexts like the online environment and the COVID-19 pandemic, were also focal points of the programme. The pace of progress to achieve full equality and fundamental rights across the EU underscored the continued relevance of the programme’s objectives. Faced with external factors such as the COVID-19 pandemic, the programme showed its flexibility by adjusting its procedures, timelines and activities.
The REC programme effectively targeted marginalised and vulnerable groups, such as victims of violence, ethnic minorities, and children, aligning with the critical societal challenges faced within the EU. Actions and priorities were aligned to beneficiaries’ diverse needs. The strong demand for support was confirmed by the high application rates under various specific objectives. The programme’s focus on capacity building, knowledge sharing, and structural support effectively addressed the requirements of its beneficiaries and provided long-lasting results, beyond the end of the projects. REC provided funding that matched beneficiaries’ strategic goals, avoiding the need for disproportionate adjustments to their activities.
Some areas of improvement suggested by beneficiaries were more attention to intersectionality in addressing gender and social inclusion issues; greater focus on emerging issues like disinformation, mental health and early childhood development and inclusion of non-EU countries to enhance international cooperation.
5.2.Europe for Citizens programme
Effectiveness
The EfC programme achieved its general and specific objectives; it largely exceeded the targets referenced in the key performance indicators. However, the achievement of the objectives is likely to have been uneven as applicants from some Member States were markedly more successful than others in tapping into this funding. The EfC programme supported projects which generated results with potential for long-term impacts, such as: (i) opportunity for young people to engage with the EU; (ii) increased fundraising and operational capacity of participating organisations; (iii) creation of sustainable outputs and project results which can be re-used and disseminated; and, (iv) contribution to sharing knowledge, raising awareness and empowering stakeholders dealing with specific topics under the thematic scope of the programme. Networking, the low administrative burden, inclusivity and the broad thematic scope were the programme’s success factors. Nevertheless, the programme was mainly visible to larger, well-networked eligible organisations with a strong focus on seeking funding opportunities. While actual beneficiaries were satisfied with the amount of information available to them, they highlighted ways in which they thought the programme could have been promoted more proactively, e.g. through social media and using interactive media. The strengths of the programme included the fact of targeting ‘ordinary’ citizens, multiannual priorities and the NCPs. Stakeholders consulted for this evaluation considered the aim of reaching through a high number of participants from different backgrounds and social groups and noted a good mix of beneficiaries and target groups. Multiannual priorities benefited the stability of the programme.
Efficiency
The benefits of the EfC programme outweighed the costs to beneficiaries despite the high level of rejection of quality proposals relative to the funds available. The simplification measures improved accessibility to CSOs of all types and sizes. Although it was challenging for some smaller or grassroots CSOs, the co-financing requirement was accepted as necessary and conducive to the leveraging of additional funds. The application and reporting processes did not cause any undue burden and were one of the strengths of the programme as they allowed for the inclusion of smaller organisations and towns/municipalities – target groups which were found to consider EU funding processes too complex otherwise. The programme’s management was positively assessed and contributed to the effective implementation of the programme.
Coherence
The EfC programme was coherent with wider EU policies and priorities. Projects funded under the programme addressed critical issues such as fake news, media literacy, and e-democracy, aligning with the EU priorities. During the COVID-19 pandemic, increased digitalisation further underscored this alignment. The programme facilitated bottom-up discussions and activities around significant EU priorities and complemented other EU programmes. The priorities it addressed included climate change, the European Green Deal, fundamental rights, and gender equality, promoting active citizenship and understanding of EU values. The programme complemented other EU programmes such as Erasmus+, Creative Europe, and the European Social Fund, supporting unique activities that enhanced its distinct contribution to EU goals. The focus on cultural heritage and remembrance activities aligned with initiatives like the European Year of Cultural Heritage, highlighting the programme’s role in promoting a shared European identity. These efforts collectively contributed to the EU overarching goals of fostering democratic participation, social cohesion, and informed citizenship.
EU added value
It is likely that the impacts identified through the EfC programme would not have been achieved without the programme’s specific EU-level support. The programme provided added value by funding cross-border activities that other funding mechanisms, particularly at national or regional levels, could not support. This was evident in its backing of European remembrance and democratic engagement, which were crucial for fostering European cultural diversity, promoting civic participation, and encouraging mutual understanding across borders. These goals aligned closely with the EU broader political agenda and societal needs, demonstrating a clear need for EU intervention. Without the EfC programme, many beneficiaries would have struggled to implement projects on a similar scale or with the same impact. For instance, the programme enabled organisations to conduct projects that spanned multiple countries, a feat they could not have achieved with purely national or regional funding. The programme’s funding was crucial for activities requiring cross-border cooperation and a broader European perspective, as no similar national or regional schemes could match the programme’s scope and scale. It complemented other EU funding sources, creating valuable synergies without overlap and enhancing the overall impact and reach of funded activities. The EfC programme contributed to establishing a level playing field across national levels by addressing diverse challenges through its transnational and cross-border approach and supporting the implementation of activities in all EU Member States. However, there were large discrepancies between the national coverage achieved in the different Member States. While the initial objective of the programme’s management was to achieve a uniform coverage across the EU, during the implementation it became apparent that the appetite to participate in the programme varied across the Member State, which explains the discrepancy (and suggests that it was out of the control of the programme’s management). The programme funded activities that involved citizens and organisations from multiple participating countries, fostering a sense of European identity and belonging. This cross-border dimension was crucial for activities such as remembrance projects, town-twinning, and pan-European networks, which helped broaden perspectives and enhance mutual understanding across Europe. The EfC programme’s ability to bridge national differences and promote a cohesive European approach was instrumental in addressing the varied challenges faced at the national level.
Relevance
The alignment with broader EU policy goals and priorities, and the fostering of a sense of European identity that were achieved by the EfC programme remain relevant objectives. The programme fostered a sense of European identity through civic engagement activities and projects focused on European remembrance, contributing to a shared understanding of European history. The European Commission’s efforts to raise awareness and improve ECI’s accessibility demonstrated a practical approach to citizen empowerment. The programme also encouraged civic participation through grassroots initiatives, fostering a sense of belonging and mutual understanding among Europeans and providing platforms for meaningful dialogue and action on common issues. Town-twinning activities promoted cross-border interactions and cultural exchanges, fostering European identity and ongoing dialogue on integration and shared values. The programme was relevant to its final beneficiaries, and mostly addressed the needs and target groups in thematic areas that remain relevant today. Activities under various strands directly contributed to enhancing citizens’ understanding of the Union’s history and diversity, fostering European citizenship, and promoting civic and democratic participation at the EU level. The programme’s structure supported a broad range of initiatives that aligned with the needs and priorities of CSOs, allowing them flexibility to tailor projects to specific thematic areas.
5.3.Citizens, Equality, Rights and Values programme
Effectiveness
The CERV programme is progressing well towards its objectives, with most results recorded over the 2021-2023 period far surpassing the milestones and targets set in the programme performance monitoring framework, in particular, in relation to the number of people expected to be reached. Between 2021 and 2023, 3 033 CSOs were reached by support and capacity building activities across all Member States. And overall, projects awarded in 2021-2023 are expected to engage at least 44 million people. Projects are contributing to programme results in a sustainable way. While it is too early at mid-term to make definitive judgements on the programme’s success and impact, the high degree of competition, the high level of continuity of projects, as well as a close alignment between call documents and EU priorities hint at the fact that projects are contributing sustainably to achieving the programme’s objectives. There was consensus across all stakeholders consulted, that the re-granting mechanisms under CERV addressed important needs in the civil society sector.
The very high ratio of quality projects is to the benefit of the fulfilment of policy objectives. However, a funding gap on the strands for equality, rights and gender equality, and for combating violence against women and children is present and may limit the ability to fulfil the objectives of these CERV strands.
The geographical balance has improved under the CERV programme with a higher proportion of projects in Eastern Europe, if compared to the predecessor programmes.
NCPs and the services that they provide to potential applicants are a key strength of the programme. Nevertheless, their delay in appointment or absence in some EU Member States has not led to an underrepresentation of beneficiaries from those Member States so far.
The programme has made a strong contribution to promoting gender equality. This is linked to financial contributions, with every fourth euro of financing from grants having contributed strongly to gender equality. Around half of the grants funded projects closely intertwined with the promotion of gender equality.
External factors, such as COVID-19, Russia’s war of aggression against Ukraine, and specific national contexts, have had a limited effect on the programme.
Efficiency
The benefits of the CERV programme outweigh the costs to beneficiaries, which are predominantly linked to the time and human resources required of applicants to invest in the application process, with large variability in the extent to which these costs were experienced between smaller organisations with limited capacity and a lower level of professionalisation of fundraising (compared with larger organisations with dedicated bid teams). By contrast, reporting generally did not appear to generate any significant costs and compared favourably to other EU funding programmes.
The main benefits of the programme are directly project-related (i.e. the opportunity to implement a desired approach) but there are also broader and societal benefits. Broader benefits include increased networking and collaboration across the EU, and enhanced visibility and recognition of organisations which have received CERV funding. Based on the evidence collected through the supporting study, societal benefits come from providing the opportunity to test innovative approaches to address societal challenges, expand target groups of projects, and improve organisations’ capacity for advocacy actions. However, the cost-of-living crisis has led to a drop in programme benefits experienced by beneficiaries, as planned project budgets were difficult to maintain, and planned activities needed to be reduced.
The new features introduced under the CERV programme have significantly improved the predictability of the programme and programme monitoring, and significantly reduced the administrative and reporting burden, thus enhancing simplification. These new features have included multiannual work programmes, lump sums and the streamlining of indicators. Reducing the reporting burden has, for example, allowed a greater results-orientation of projects. The introduction of unit costs at Commission level, however, appears to have added complexity, exacerbated by the fact that these unit cots do not reflect the actual costs of activities and impose unrealistic limitations on travel costs. The implementation of the monitoring framework, including dedicated data collection tools, is a clear improvement compared with the two predecessor programmes. The introduction of re-granting mechanisms, a novel approach in this policy area, has helped to improve accessibility of the programme for smaller, less experienced beneficiaries through simplified application processes facilitated by intermediaries.
Several adjustments could support greater efficiency and improve the programme’s overall effectiveness, including a more user-friendly eGrants system and more encouragement to exploit digital tools; and a more streamlined EU Survey on Justice, Rights and Values, which is the Commission already revised in parallel to this evaluation.
Coherence
The CERV programme occupies a space in the CSO funding landscape that would otherwise be vacant. Its comprehensive range of themes, pan-EU coverage, specific focus on EU values, grants of a significant size, operating grants and re-granting mechanisms, all make the programme an essential source of funding for organisations promoting EU values and rights. One of the key advantages of the CERV programme is its ability to provide independent sources of financing, allowing organisations to maintain their autonomy and pursue their goals without undue influence. Nevertheless, as the funding gap is significant, beneficiaries, if they can, also use other sources of funding, such as, for example, the Visegrad Fund or other EU programmes, if they intersect with the CERV programme’s topics and if synergies are possible.
On the internal coherence, the move to the single architecture with work plans and call planning under a single umbrella has worked satisfactorily. Potential applicants and beneficiaries generally understand the concept of strands, appreciate having a ‘one-stop’ shop and consider that this has led to greater coordination at EU level.
EU added value
CERV is very often the only source of funding available to CSOs working in the areas it covers. EU funding plays a major role in supporting civil society across the EU, and the CERV programme is achieving results that can only be achieved by action at EU level. Member States are not now and are not likely to step in to fund work on rights and EU values to any significant extent, if at all. In some cases, there are national funds available, but the decision-making process is perceived often to be subject to political influence or to take a narrow national focus.
The effects of having no CERV programme would be negative and likely impact the already shrinking civic space in the EU. Respect for the rule of law, fundamental rights and democratic dialogue, gender equality and disability rights would also be affected. Violence against children, extremism and radicalism would be likely to rise. The sense of being a European citizen would weaken. These conclusions are strongly supported by the evidence from consultation of stakeholders and experts as well as desk research on relevant trends across the EU Member States.
Relevance
The programme remains relevant in view of its objectives as set out in the Regulation. Recent evidence confirms the gaps identified in the impact assessment as regards citizens identifying as being European, their awareness of recent history, participation in political and social activities, the level of awareness and respect of the rule of law, democracy and fundamental rights in many EU countries, the persistence of gender gaps, continued discrimination on the basis of gender identity, race or ethnicity, sexual orientation, religious background, etc., as well as violence against women and children. Many of these gaps have been further amplified by challenges such as the COVID-19 pandemic and the economic downturn that followed, as well as Russia’s war of aggression against Ukraine.
There is a very strong alignment between the needs of the target stakeholders and the actions funded under the programme, not only in terms of the actions responding directly to beneficiary organisations’ missions and strategies, but also allowing them to pursue EU added value by implementing projects across multiple EU Member States. The programme addresses important needs of CSOs working across the EU, thus contributing to the EU’s capacity to respond to the above-mentioned challenges. It has allowed the implementation of projects for which no comparable funding (in terms of grant size and EU-wide dimension) would have been available otherwise.
The programme has already demonstrated flexibility to adapt to changing needs, both in terms of processes and procedures to respond to the COVID-19 public health restrictions, to consequences from Russia’s was of aggression against Ukraine as well as thematically.
5.4.CERV: operational findings and lessons learned
Based on the conclusions of the interim evaluation, some areas for improvements or follow-up could be addressed through the current implementation of the CERV programme.
The programme showed an excellent performance over the 2021-2023 period, with most results significantly exceeding the milestones and targets. It may be necessary to reassess and revise the targets set in the programme performance monitoring framework to better reflect the programme’s actual capacity and impact.
The re-granting mechanisms demonstrated their effectiveness in addressing critical needs within the civil society sector and for the programme target groups. Besides the continuation of this scheme, it may be worthwhile considering its potential expansion and scaling up to further support the sector and generate an even greater impact.
The CERV programme has already reached a large number of people. However, it is primarily well-known among larger organisations with EU-wide networks, suggesting that its reach and recognition could be improved among a broader audience. To address this, implementing more targeted communication measures could help increase the programme’s visibility and awareness beyond its current scope.
The programme’s significant contribution to promoting gender equality is a notable achievement. This success could be highlighted as a best practice and showcased as a model for other programmes to follow, demonstrating effective ways to promote and advance gender equality – and equality overall – through strategic funding and initiatives.
There remains scope to further capitalise on opportunities brought on by digitalisation at activity level. Despite the possibility provided by the work programmes, these elements do not appear to have yet generated interest or capacity to respond to the opportunities presented, with only a small number of ICT tools funded thus far compared with other types of activities.
Annex I. Procedural Information
Lead DG, DECIDE Planning/CWP
This evaluation was included in the Commission’s agenda planning system (PLAN/2023/1854) and carried out in compliance with the Article 15(4) of the
EfC Regulation 390/2014
, Article 13(4) of the
REC Regulation 1381/2013
, and Article 17 of the
CERV Regulation 2021/692
. In line with the
Better Regulation Guidelines
, it assesses the extent to which the three programmes were effective, efficient and coherent, provided EU added value and remained relevant to tackle present needs.
The evaluation was carried out by the Directorate-General for Justice and Consumers (DG JUST) in collaboration with the European Education and Culture Executive Agency (EACEA).
DG JUST carried out the ex post evaluation of the REC programme based on a two-step approach to both fulfil the requirements of the Regulation and carry a meaningful, cost-effective evaluation exercise. The first part of the ex post evaluation was completed in March 2022; it provided an overview of the funding distribution and assessed the preliminary achievements of the programme, which fed into the design of the activities of the CERV programme. The second part instead focused on assessing long-term impact and sustainability. The structure and timeframe of the evaluation were agreed with the Secretariat-General and Legal Service. Agreement was re-confirmed by the dedicated Inter-Service Support Group supporting the evaluation during its meeting of 4 October 2024.
Organisation and timing
DG JUST prepared the evaluation
roadmap
, the stakeholder consultation strategy and the technical specifications for the supporting study contract. The documents were submitted for feedback and approval to the dedicated Inter-Service Steering Group (ISSG).
The ISSG was established in June 2023 following the invitation sent on 25 May 2023 to the following DGs: BUDG, HOME, COMM, SANTE, RTD, DIGIT, EAC, EMPL, ESTAT, CNECT, REGIO, ENV, REFORM, INTPA, NEAR, the Secretariat-General, the Legal Service and the European External Action Service. The group met five times (see below ‘Evidence, sources and quality’).
DG JUST conducted the external supporting study with external experts between 5 December 2023 – 18 November 2024. The ISSG was consulted on the supporting study report (inception, interim, final and synopsis reports) during the dedicated meetings and through a dedicated online collaborative space. The relevant evaluation documents (evaluation roadmap, call for evidence, open public consultation) were published on the
Europa ‘Have your Say’ dedicated page
for stakeholders’ feedback and consultation respectively between 22 August 2023 and 19 September 2023 for the call for evidence, and between 4 April 2024 and 27 June 2024 for the public consultation.
DG JUST concluded the evaluation in 2025 with issuing a Commission Report and a stakeholders consultations synopsis report.
Exceptions to the Better Regulation Guidelines
The Commission’s
Better Regulation Guidelines
were followed to carry out the evaluation without deviations.
Evidence, sources and quality
The table below summarises the successive steps of the evaluation. An external supporting study was carried out a consortium led by Tetra Tech International Development, with a contract duration of 50 weeks. The study applied a mix of evaluation methods including desk research, online public consultation, surveys, focus groups, workshops and interviews with stakeholders, EU officials and Member States’ representatives. The ISSG concluded that the study was conducted in line with the technical specifications and the agreed inception report. The study used relevant qualitative and quantitative sources and methods, although data had some specific caveats (discussed in Section 1.3 and Annex II). The analysis and conclusions are sound, and the methodological framework and its limitations are clearly outlined.
Table 1: Evaluation timeline
|
Steps/tasks
|
Timing
|
|
Preparation (March 2023 – November 2023)
|
|
Draft the evaluation roadmap and the consultation strategy
|
March – May 2023
|
|
Set up the ISSG and draft the supporting study technical specifications (ToR)
|
May – June 2023
|
|
ISSG meeting to discuss the ToR and the consultation strategy
|
22 June 2023
|
|
Tendering procedure
|
July – November 2023
|
|
Roadmap publication
|
August 2023
|
|
Call for Evidence
|
22 August 2023 – 19 September 2023
|
|
Evaluation supporting study (December 2023 – November 2024)
|
|
Signature of the contract
|
5 December 2023
|
|
Kick-off meeting
|
11 January 2024
|
|
Inception meeting (ISSG)
|
14 February 2024
|
|
Online public consultation
|
4 April 2024 – 27 June 2024
|
|
Interim meeting (ISSG)
|
5 July 2024
|
|
Validation workshop
|
19 September 2024
|
|
Final report meeting (ISSG)
|
4 October 2024
|
|
Finalisation of the supporting study
|
October – November 2024
|
|
Report and Staff Working Document (November 2024 – June 2025)
|
|
Draft Report, SWD and Synopsis report
|
November – December 2024
|
|
ISSG meeting on Report and SWD
|
22 January 2025
|
|
Prepare and launch the Inter-Services Consultation package
|
February 2025
|
|
Revision and presentation to College of final Report
|
May 2025
|
|
Transmission of the Commission report to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions (Interinstitutional Database, Europa, etc.)
|
June 2025
|
Source: DG JUST
Consultation of the Regulatory Scrutiny Board (RSB)
The evaluation was not scrutinised by the RSB.
Annex II. Methodology and Analytical models used
This annex provides an overview – complementary to Chapter 1 – of the methodology adopted to carry out this evaluation. It details data collection methods and tools used, as well as the limitations to the reliability of the analysis carried out.
II.1 Methods and tools
This Staff Working Document builds on data collected and findings obtained through the supporting study carried out by the contractors. The study provided an evidence-based assessment of the relevance, effectiveness, efficiency, coherence and EU added value of the three programmes. An intervention logic was designed or reviewed for evaluating each programme. The supporting study comprised of a mixed-methods approach utilising quantitative and qualitative data collection tools and was underpinned by extensive desk research. Five case studies were also conducted for the interim evaluation of the CERV programme. The figure below presents and overview of the study’s methodology.
Figure 30. Overview of study methodology
The following sections provide an overview of the methods used in the supporting study.
Study desk-based research (secondary data collection)
The desk research encompassed a wide range of available, relevant documents at policy, programme and project level for all three programmes. The policy review was based on policy-specific reports as well as political and policy documents to support the assessment of coherence and complementarity of the programmes with EU policies or other funding programmes with similar objectives. The review also helped to show interlinks between the programmes and their wider policy context. At programme level, programming documents for all three programmes were reviewed to support the analysis of questions under effectiveness, efficiency, and relevance, as well as to contribute to collecting and aggregating data on specific indicators listed for each programme. Additionally, the study also conducted a literature review to identify any available studies on the effects of the programmes (based on a relevant selection of academic and grey literature databases and search engines). Furthermore, the study prepared data from the special programme’s Eurobarometers. For project-level documents, an automated text mining approach was used to extract relevant data from proposals and final reports as follows:
•For REC – Text mining was carried out on a sample of 339 final reports of grants funded in 2020 to extract information on the results reported, based on a typology of results developed as part of the first part ex post evaluation. The results were added to the text mining carried out by the first part ex post evaluation on a sample of 211 grant reports. In addition, text mining was carried out on the sample of final reports, to extract information on the main target groups reached.
•For EfC – Text mining was carried out on a large sample of final reports of action grants to identify impacts such as policy initiatives following-up on activities supported by the programme at the local or European level, examples of collaboration between beneficiaries in similar events or projects, and examples of transfer of good practices.
•For CERV – Text mining was carried out to extract the number of people reached by training and mutual learning and exchange of good practices activities in final reports, to allow for a comparison with number of people estimated in Part C data. Text mining also served to conduct a qualitative analysis of evaluator feedback on 1 284 proposals to assess commonalities between the four gender scores as part of case study 1.
The desk review also included project-level data collection, including: grant data, procurement data, Part C data, results of the EU Survey on Justice, Rights and Values.
Study fieldwork activities (primary data collection)
This task comprised the stakeholder consultation activities conducted for the evaluation. A broad range of stakeholders were reached through the various consultation activities, covering a wide geographical scope, showing views of all stakeholder groups directly affected by or having an interest in the programme. The consultation activities are summarised below and analysed in the stakeholders’ consultation synopsis report (Annex V).
For the quantitative data collection, the study included five consultation exercises and surveys – a public consultation, a targeted survey of beneficiaries, a targeted survey of unsuccessful applicants, a survey of NCPs, and a beneficiary survey on costs. A sixth survey was conducted as part of case study 3 of recipients of funding via intermediaries (so-called third-party beneficiaries).
Online public consultation
The online public consultation aimed to strengthen the evidence base for the evaluation of the three programmes by gathering the views of relevant stakeholders on the programmes’ effectiveness, efficiency, relevance, coherence, and EU added value. It was launched on 4 April 2024 and was open for 12 weeks (closing on 27 June 2024). The questionnaire – including closed and open questions – was published on the
dedicated “Have your say” webpage
of the European Commission. Respondents were also invited to submit position papers at the end of the survey. A total of 53 responses were received. The factual summary report can be consulted on the
dedicated “Have your say” webpage
. As indicated in the summary report, responses to the consultation cannot be considered as a representative sample of the views of the EU population.
Targeted survey of beneficiaries
The survey of beneficiaries was shared with all EfC project coordinators for the entire programming period 2014-2020, all REC project coordinators for whom contact details were available for the same period, and all CERV project coordinators who had received funding from 1 January 2021 until 31 December 2023. It was hosted on the EU Survey platform. To limit the burden on respondents, each contact only received an invitation to participate in one of the surveys by deduplicating beneficiaries and unsuccessful applicants of the three programmes. The questionnaires were translated into all EU-27 official languages, as well as Albanian, Bosnian, Macedonian, and Serbian, to reflect the languages used in non-EU countries that were eligible for the Europe for Citizens programme. The survey was launched on 5 March and closed on 26 March 2024. It was shared with 2 583 beneficiaries in total (EfC = 1 480, REC = 427, CERV = 676) and received 421 responses, thus achieving a 16% response rate. The response rate and number of respondents was considered sufficient to have produced valid results. Respondents came from all EU-27 Member States, as well as Albania, Bosnia and Herzegovina, North Macedonia, Serbia, and the United Kingdom. The survey enabled the collection of data suitable for statistical analysis such as on recurrent beneficiaries.
Targeted survey of unsuccessful applicants
The survey of unsuccessful applicants was launched on 4 March and closed on 25 March 2024 for those who had unsuccessfully applied for REC and CERV. To collect responses from unsuccessful applicants of the EfC programme, the survey was re-launched on 6 May and closed on 24 May. It was hosted on the EU Survey platform. After deduplication to include only organisations that had never received a REC/EfC/CERV grant before, the survey was shared with 4 664 unsuccessful applicants in total (EfC = 1 641, REC = 1 029, CERV = 1 994) and received 343 responses, thus achieving a 7% response rate. The response rate and the number of respondents was considered sufficient for having produced valid results, particularly considering the challenge of engaging unsuccessful applicants, for whom the time investment in responding to a survey about programmes that they had not been a beneficiary of could reasonably be assumed to be inefficient. Respondents came from all EU-27 Member States, as well as Israel, Kenya, Montenegro, Mozambique, Norway, Serbia, and Tanzania.
Survey of NCPs
A dedicated survey targeted all NCPs for EfC and CERV in 21 EU Member States. The survey opened on 13 March 2024 and closed on 19 April 2024, and it was hosted on the EU Survey platform. Overall, 14 NCPs responded to the survey. The survey did not ask about the country in which they were based to ensure anonymity of responses.
Beneficiary survey on costs
A short survey targeted at CERV beneficiaries sought to collect data on average costs of activities. It received 97 responses from beneficiaries in 19 EU Member States. However, findings were treated with a great level of caution because the evaluation could not check the methodology that respondents applied when providing their estimates. This is the reason why the analysis by country of the indicator on average costs of actions (cf. 4.1.2 Efficiency, CERV programme) only included findings related to countries with four or more responses.
Survey of third-party beneficiaries
A targeted survey of third-party beneficiaries (beneficiaries of grants re-granted through intermediaries under the CERV programme) was conducted as part of case study 2 on simplification efforts. Overall, 212 respondents participated in the survey from 10 EU Member States (Bulgaria, Croatia, Cyprus, Denmark, Greece, Hungary, Lithuania, Poland, Slovenia and Spain). The response rate and number of respondents was considered sufficient to have produced valid results. 66% of respondents (147 of 212) reported that they would describe their organisation as a ‘grassroot organisation’, with organisations working in the fields of democracy, equality, fundamental rights, human rights and rule of law.
The qualitative data collection involved interviews with beneficiaries (n=227 across all three programmes), focus groups with NCPs (n=15, EfC and CERV only), focus groups with Programme Committee Members (n=13 across all three programmes) as well as 44 deliberative workshops with EU networks, national stakeholders and citizens (n=229, CERV only).
Interviews with beneficiaries (and other stakeholders)
The supporting study conducted 227 interviews with beneficiaries across all three programmes (EfC = 80, REC = 50, CERV = 97). The sample of interviewees covered all EU-27 Member States across the three programmes, based on project coordinators’ countries, as well as Serbia. The sample aimed for a proportionate approach to balance types of grants, types of beneficiaries per programme, strands, grant size, years when grant agreements were signed, recurring beneficiaries. Interview guides were tailored to each programme and grant type. The interviews were organised in five waves, each lasting two weeks. During the first wave, interview guides were piloted on a sample of 40 interviews. Following the pilot, the guides were further refined before deploying to a team of national researchers to conduct interviews in all EU-27 Member States and Serbia.
Additionally, the supporting study conducted six expert interviews covering thematically the four strands of CERV. These included independent experts as well as two members of the European Cooperation Network and the EU Network on children’s rights. The aim of these interviews was to validate early results and conclusions, particularly as regards relevance and EU added value of the CERV programme.
Finally, a total of 11 interviews were conducted by the study team with European Commission (9) and EACEA (2) officials.
Focus groups with NCPs
Two online focus groups with NCPs were conducted on 21 March 2024 and 4 April 2024. NCPs from 10 EU Member States were invited to participate, namely Cyprus, Czechia, Estonia, France, Italy, Latvia, Netherlands, Portugal, Romania, and Sweden. They were selected based on geographical balance, country size, type of organisation, and whether NCPs were part of the previous EfC network. Overall, NCPs from eight EU Member States participated in the focus groups.
Focus groups with Programme Committee Members
Three online focus groups with 13 Programme Committee Members were held on 5 April 2024 (REC), 10 April 2024 (EfC), and 26 March and 9 April 2024 (CERV). Only one former REC Programme Committee Member attended the focus group (Finland). In terms of former EfC Programme Committee Members, three were consulted via the focus group (Finland, Malta) and one finally via a short interview (Greece). There were nine participants in the two focus groups with CERV Programme Committee Members, two of which were also former EfC and REC Programme Committee Members (Austria, Croatia, Estonia, Finland, France, Germany, Greece, Romania, Slovenia, Spain).
Deliberative workshops
The study team conducted 44 deliberative online workshops with stakeholders and citizens focusing on Strand 2 and Strand 3 of the CERV programme.
The workshops with EU networks and national stakeholders aimed to collect insights on: (1) the most pressing needs that organisations faced in their country and the EU more broadly, (2) the national and EU funding landscape for organisations working in specific policy areas, (3) stakeholder perceptions of CERV in terms of the clarity of the set-up and requirements, eligibility, satisfaction with topic areas covered and activities funded, suitability of programme objectives, and gaps, (4) the design of an ‘ideal’ EU funding programme. Out of the 40 EU networks invited to the four workshops on children’s rights, disability, gender equality, and non-discrimination, 17 participated. Out of the 300 national stakeholders contacted, 65 participated in the 20 deliberative online workshops on citizens’ engagement and participation, covering 6 EU Member States in total.
The workshops with citizens aimed to understand their general interests in the topic areas, previous participation in events (if any), preferred types of activities, as well as views on selected CERV projects that were presented in the workshops. Insights on citizens’ preferred ways of receiving communication about EU-funded projects were also collected. The 30 online workshops focused on three thematic areas: EU Remembrance, citizens’ engagement and town twinning / networks of towns. Invitations to register for the workshops were shared on relevant social media groups (in particular, those bringing together residents of local areas), as well as through national stakeholders who participated in other workshops. Citizens were offered EUR 30 (or equivalent) as an incentive for their participation, in the form of a gift voucher to a popular online shopping platform in their country. Overall, 164 stakeholders were consulted in 10 EU Member States.
Case studies
The five thematic case studies explored and provided additional in-depth qualitative and quantitative evidence illustrating CERV topical issues. While each drew on findings from the main data collection activities conducted in the supporting study, for three case studies, additional data collection was conducted. Table 2 below provides an overview of the case studies’ themes and methodology.
Table 2. Overview of the thematic case studies
|
Case study
|
Methodology
|
|
Case study 1: Gender equality perspective and (prospective) impacts on gender equality at the level of the CERV programme and its activities
|
Document review of relevant programme and policy documents; findings from main data collection activities; four interviews with evaluators of CERV project proposals about their experience of applying the gender mainstreaming methodology; in-depth review of 120 out of 1 284 project proposals, which were sample proportionately by gender score and strand (17 under Union values, 38 under Equality, rights and gender equality, 41 under Citizens’ engagement and participation, 24 under Daphne); expert review of call documents of calls for proposals to promote gender equality (2022, 2024) and calls for proposals to prevent and combat gender-based violence and violence against children (2022, 2023 and 2024); in-depth review of all remaining call documents published under the 2021-2022 and 2023-2024 work programmes that are available on the Funding and Tenders portal; qualitative analysis of evaluator feedback on 1 284 project proposals to assess commonalities between the four gender scores, which was conducted using text mining and manual cross-checks; in-depth review of final project reports for which gender scores were available (n=101) as regards reported results and impact.
|
|
Case study 2: Effects generated by simplification efforts
|
Review of legislative and regulatory documents, programme reports, evaluation studies; feedback from stakeholders, including beneficiaries, EU officials, collected as part of the main data collection activities.
|
|
Case study 3: Adaptability and suitability of the CERV programme to respond to the (possibly combined) impacts of COVID-19 pandemic and new digital technologies
|
Desk research of programme and project documents; results from stakeholder feedback collected as part of the main data collection activities.
|
|
Case study 4: Support to CSOs engaged in Union values strand and effects of promoting EU values
|
Findings from main data collection activities; three interviews with beneficiaries of the calls for proposals to promote CSOs’ awareness of capacity building and implementation of the EU Charter of Fundamental Rights (i.e. CHAR-LITI calls); survey of third-party beneficiaries.
|
|
Case study 5: Synergies with other funding programmes
|
Findings from main data collection activities; mapping exercise of 12 EU programmes and 13 non-EU funds.
|
Financial data
The key challenges related to REC programme’s data comparability are presented in Section 1.2 of this Staff Working Document. This annex provides a complementary methodological discussion on data limitations and explains the mitigation measures undertaken.
Significant efforts were made to provide an accurate picture of the information on the state of play and indicators of the REC programme. The dataset used to analyse data for the first part ex post evaluation needed to be completed with grant agreements that could not be included at that time. The dataset was complemented with data from the Commission’s Accrual-based Accounting (ABAC) system so to update the most important indicators. As a result, some indicators provide a full picture of implementation across the 2014-2020 period (covering all grants and procurement funded), while other indicators are still based on a subset of grants. However, this ‘sample’ approach is considered appropriate for the indicators to which they relate.
In addition, limitations exist with regard to comparability of the findings resulting from the data from the text mining carried out on final reports for the first part ex post evaluation, and for this evaluation. This relates to the key results and number of target groups reached through different activities. While the same categories were used for the text mining as in the first part ex post evaluation, a detailed methodology used for the text mining carried out as part of the first part ex post evaluation was not available. Hence both exercises may have searched for a different set of terms under each category. While the data on the key results reported were of a similar proportion and magnitude for both evaluations (and therefore have been combined, but should be treated with suitable caution nevertheless), the results of the extraction of target groups reached by activities were of a different magnitude. Consequently, they have been reported on by evaluation question.
Finally, the following limitations arise in relation to 2014 and 2015 REC grant data due to the phase-out of the Commission’s application system Priamos:
(1)it was not possible to verify data on 331 applications, which was missing the information on the year applications were submitted. As a result, comparison of the number of applications and grants awarded (by annual work programme, per year) could not be provided for the full period 2014-2020. However, this information is available for 2016 onwards.
(2)the data on the number of applications and the amount requested under the specific objective RDIB (to promote and protect the rights of persons with disabilities) is incomplete, as the number of applications for 2014-2015 was not available. As a result, success rates for this specific objective could not be calculated (unlike for the other specific objectives).
(3)data on the country of the coordinator was not available for 135 grants. Therefore, the analysis shows the distribution of grants awarded and budget committed by coordinator country for 815 of the 950 REC grants awarded. This also meant that the success rate by country could not be calculated, both for this reason and because, as noted above, the data on the number of applications is incomplete as well.
Data analysis and triangulation
Data analysis included both qualitative and quantitative methods based on the data collected in relation to each evaluation question. The results stemming from the various data collection activities and analysis were triangulated to develop robust answers to the evaluation questions. Several triangulation methods were used, including: triangulation of different types of data, data collection tools, types of stakeholders as well as data sources. Triangulation was carried out for each evaluation question separately.
Finally, the supporting study team organised a validation workshop with DG JUST and EACEA aiming at discussing the draft conclusions and lessons learned.
II.2 Points of comparison
The points of comparison presented in the tables below were established based on core performance indicators established by the respective regulations, previous evaluations and impact assessments as well as performance monitoring frameworks. Explanations are included under Section 2.3.
Table 3. REC programme: objectives, indicators and points of comparison
|
Indicator
|
Points of comparison (baseline)
|
Points of comparison (target per year, 2020)
|
Achievement by end of programme
|
|
General and specific objectives (SO)
|
|
Female employment rate 20-64 age group (also SO 4)
|
62.4% (2012)
|
71%
|
66.2% (2020)
|
|
Employment rate of people with disabilities (also SO 3)
|
48.5% (2013)
|
55%
|
50.7% (2020)
|
|
The gender pay gap (also SO 4)
|
16.8% (2013)
|
14%
|
12.3% (2021)
|
|
The percentage of women among non-executive directors on boards of listed companies (also SO 4)
|
16.2% (2011)
|
40%
|
36.3% (2023)
|
|
The number of Member States that set up structural coordination mechanisms with all stakeholders, including Roma, on the implementation of the National Roma Integration Strategies (SO 1)
|
0 (2013)
|
26
|
27 (2023)
|
|
Percentage of Europeans who consider themselves as ‘well’ or ‘very well’ informed of the rights they enjoy as EU citizens (SO 8)
|
32% (2010)
42% (2015)
|
51%
|
60% (2024)
|
|
Percentage of people that consider domestic violence against women unacceptable (SO 5)
|
84% (2010)
|
100%
|
96% (2016)
|
|
Consumer and retailer awareness of rights and obligations (SO 9)
|
36% (2011)
|
|
31.5% (2022)
|
|
The perception of consumers of being protected (SO 9)
|
64% (2011)
|
75%
|
76% (2022)
|
|
Applications and awards (all specific objectives)
|
|
Number of applications and grants related to each specific objective
|
0
|
|
RCHI = 364 / 82
RCIT = 184 / 26
RCON = 0
RDAP = 1 793 / 294
RDAT = 43 / 26
RDIB = 50 / 64
RDIS = 1 202 / 238
RGEN = 126 / 86
RRAC = 1 141 / 134
|
|
Geographical coverage of the activities funded by the programme (i.e. number of grants awarded by Member State)
|
0%
|
100% of the participating countries
|
100% of the participating countries.
Most awarded actions had a coordinator from Italy (128 grants awarded), followed by Belgium (106), Greece (57) and Spain (50). Other countries which were coordinators of many awarded grants (between 30 and 41 awarded grants) included: Austria, Bulgaria, Germany, France, the Netherlands and Hungary). The countries with the least grants awarded to a coordinator included Iceland (2), Denmark (3) and Serbia (4).
|
|
Budget allocation and distribution (all specific objectives)
|
|
Level of funding requested by applicants in relation to each specific objective
|
0
|
|
RCHI = 121 282 165.82
RCIT = 51 800 047.90
RCON = 0
RDAP = 602 507 355.08
RDAT = 12 882 806.10
RDIB = 3 057 680.32
RDIS = 288 289 703.09
RGEN = 113 729 208.57
RRAC = 465 895 521.89
|
|
Level of funding granted in relation to each specific objective (i.e. funding committed to grants)
|
0
|
|
RCHI = 25 589 639
RCIT = 7 048 894
RCON = 0
RDAP = 111 624 288
RDAT = 6 261 141
RDIB = 21 745 435
RDIS = 63 463 953
RGEN = 32 588 567
RRAC = 51 618 603
|
|
Common indicators to the programme
|
|
Number and percentage of persons in a target group reached by the awareness-raising activities funded by the programme
|
0
|
|
Number and percentage not available.
|
|
Number of stakeholders participating in, inter alia, training activities, exchanges, study visits, workshops and seminars funded by the programme
|
0
|
|
1 973 154
|
|
Improvement in the level of knowledge of Union law and policies and, where applicable, of rights, values and principles underpinning the Union, in the groups participating in activities funded by the programme compared with the entire target group
|
NA
|
|
Reported as key result by 57% of the funded projects (sample-based).
|
|
Number of cases, activities, and outputs of cross-border cooperation
|
0
|
|
207 (i.e. number of projects that reported the creation of mechanisms or tools for cross-border cooperation sustained after the end of the project in their reports)
|
|
Participants’ assessment of the activities in which they participated and of their (expected) sustainability
|
NA
|
|
Beneficiaries interviewed for the evaluation elaborated on the longer-term impacts that resulted from their participation in REC-funded projects. They highlighted several significant outcomes.
|
Table 4. EfC programme: objectives, indicators and points of comparison
|
Indicator
|
Points of comparison (baseline)
|
Points of comparison (target 2020)
|
Achievement by end of programme
|
|
General objective: To contribute to citizens’ understanding of the Union, its history and diversity, to foster European citizenship and to improve conditions for civic and democratic participation at Union level.
|
|
Percentage of EU citizens feeling European
|
59%
|
Stable at 59%
|
70% (2020), 72% (2023)
|
|
Specific objective 1 (Strand 1: European Remembrance): To raise awareness of remembrance, the common history and values of the Union and the Union’s aim, namely to promote peace, the values of the Union and the well-being of its peoples, by stimulating debate, reflection and the development of networks.
|
|
Number of participants who are directly involved
|
100 000
|
100 000 beneficiaries
|
1 724 932 (2014-2020)
|
|
Number of persons indirectly reached by the programme
|
150 000
|
202 500 beneficiaries
|
>60 million (2014-2020)
|
|
Number of projects
|
31
|
36
|
40 (average per year, 2014-2020)
|
|
Quality of the project applications and the degree to which the results of selected projects can be further used, transferred.
a.Lowest score obtained by a retained project.
b.Number of events organised.
|
a. 80
b. 50
|
a. 83.2
b. 95
|
a. 80
b. 245 (2019)
|
|
Percentage of first-time applicants
|
40%
|
-5% < 40% < 5%
|
61% (average, 2014-2020)
|
|
Specific objective 2 (Strand 2: Democratic and civic engagement): To encourage the democratic and civic participation of citizens at Union level, by developing citizens’ understanding of the Union policymaking-process and promoting opportunities for societal and intercultural engagement and volunteering at Union level.
|
|
Number of participants who are directly involved
|
1 000 000
|
100 000 beneficiaries
|
7 647 859 (2014-2020)
|
|
Number of persons indirectly reached by the programme
|
1 000 000
|
202 500 beneficiaries
|
>300 million (2014-2020)
|
|
Number of participating organisations
|
1 000
|
1 700
|
2 378
|
|
Perception of the Union and its institutions by the beneficiaries
|
77%
|
77%
|
70%
|
|
Quality of project applications (lowest score obtained by a retained project)
|
71
|
74
|
82 (Civil society projects)
71 (Town twinning)
81 (Networks of towns)
|
|
Percentage of first-time applicants
|
40%
|
-5% < 40% < 5%
|
61% (average, 2014-2020)
|
|
Number of transnational partnerships including different types of stakeholders
|
1.3
|
At least 2
|
6 (average number of partners per project)
|
|
Number of networks of twinned towns
|
41
|
36
|
42
|
|
Number and quality of policy initiatives following-up on activities supported by the programme at the local or European level
|
0
|
2
|
Several examples available (based on qualitative analysis of a sample of project reports)
|
|
Geographical coverage of the activities
a.Number of Member States with appropriate national coverage (NC)
b.Submitting as a lead partner
c.Selected as a lead partner
d.Submitting as co-partner
e.Selected as co-partner
|
a. 0
b. 13
c. 12
d. 18
e. 15
|
a. 25
b. 25
c. 28
d. 28
e. 28
|
a. Large discrepancies observed
b. 28
c. 28
d. 28
e. 28
|
Table 5. CERV programme: objectives, core performance indicators and points of comparison
|
Indicator
|
Points of comparison (baseline)
|
Points of comparison (target 2027)
|
Achievement 2021-2023
|
|
Objective 1: Protect and promote Union values
|
|
People reached by training activities
|
0
|
80 871
|
298 434
|
|
People reached by mutual learning and exchange of good practices activities
|
0
|
26 493
|
1 107 278
|
|
People reached by awareness-raising, information and dissemination activities
|
0
|
2 014 291
|
24 058 197
|
|
CSOs reached by support and capacity building activities
|
0
|
6 300
|
1 266
|
|
Objective 2: Promote rights, non-discrimination, equality, including gender equality, and advance gender and non-discrimination mainstreaming
|
|
People reached by training activities
|
0
|
31 852
|
143 854
|
|
People reached by mutual learning and exchange of good practices activities
|
0
|
10 434
|
194 485
|
|
People reached by awareness-raising, information and dissemination activities
|
0
|
793 347
|
10 613 026
|
|
CSOs reached by support and capacity building activities
|
0
|
847
|
327
|
|
Objective 3: Promote citizens engagement and participation in the democratic life of the Union and exchanges between citizens of different Member States and to raise awareness of the common European history (Citizens’ engagement and participation strand)
|
|
People reached by training activities
|
0
|
84 181
|
5 975
|
|
People reached by mutual learning and exchange of good practices activities
|
0
|
2 034 111
|
3 132 968
|
|
People reached by awareness-raising, information and dissemination activities
|
0
|
589 266
|
354 782
|
|
CSOs reached by support and capacity building activities
|
0
|
2 372
|
1 109
|
|
Transnational networks and initiatives focusing on European memory and heritage as a result of programme intervention
|
0
|
1 500
|
399
|
|
Objective 4: Fight violence, including gender-based violence (Daphne strand)
|
|
People reached by training activities
|
0
|
39 499
|
114 952
|
|
People reached by mutual learning and exchange of good practices activities
|
0
|
12 940
|
36 466
|
|
People reached by awareness-raising, information and dissemination activities
|
0
|
983 821
|
4 326 933
|
|
CSOs reached by support and capacity building activities
|
0
|
1 120
|
331
|
CERV programme: progress towards result, impact and context indicators
I. Progress towards general objectives against baselines
The general objective of the programme is to protect and promote rights and values as enshrined in the Treaties, the Charter and the applicable international human rights conventions, in particular by supporting CSOs and other stakeholders active at local, regional, national and transnational level, and by encouraging civic and democratic participation, in order to sustain and further develop open, rights-based, democratic, equal and inclusive societies which are based on the rule of law.
Indicator 1: Number of Member States having a Contact Point
·Target 2027: 27
·Baseline: 0
·Results: 14 (2021), 18 (2022), 19 (2023)
·Overall assessment: Achievement of this indicator will depend on the remaining Member States’ impetus for establishing NCPs.
Indicator 2: Citizens satisfied with how democracy works in the European Union
·Target 2027: Increase (unspecified) in those who reported being ‘satisfied’ with how democracy worked in their country.
·Baseline: 41% (EU Survey on Justice, Rights and Values – event survey, 2022), 54% (Eurobarometer 2019).
·Results: 40% (event survey, Q3 2023), 57% (Eurobarometer spring 2024).
·Overall assessment: Barring any significant political changes in EU Member States until 2027 that could put citizens’ satisfaction with how democracy works at risk, this indicator is on track to increase by 2027.
Indicator 3: Feeling like a citizen of the European Union
·Target 2027: Increase (unspecified) in those who reported ‘yes, definitely’ or ‘yes, to some extent’ about feeling like a citizen of the EU.
·Baseline: 87% (EU Survey on Justice, Rights and Values – event survey, 2022), 70% (Eurobarometer 2022).
·Results: 86% (event survey, Q3 2023), 74% (Eurobarometer spring 2024).
·Overall assessment: In terms of the event survey, this indicator is on track to achieve an increase by 2027. As regards the Eurobarometer, there has been a 4-percentage point increase in 2024 against the baseline, which allows for the assumption that an increase in this indicator by 2027 might be achieved.
Indicator 4: Respect of the core values of the EU
·Target 2027: Increase (unspecified) in those who ‘totally agreed’ or ‘tended to agree’ that the core values of the EU, such as fundamental rights, democracy and rule of law were well protected in their country.
·Baseline: 61% (EU Survey on Justice, Rights and Values – event survey, 2022), 64% (Eurobarometer 2022).
·Results: 61% (event survey, Q3 2023), 65% (Special Programme Eurobarometer 2021), 66% (Special Programme Eurobarometer 2024).
·Overall assessment: Barring any significant political changes in EU Member States until 2027 that could negatively affect citizens’ perceptions of the respect of core values of the EU in their country, this indicator is on track to increase by 2027. It has already increased vis-à-vis the 2022 baseline.
Indicator 5a: Awareness of the rule of law among the general public – in their own country
·Target 2027: Decrease (unspecified) in those who feel not very well informed or not at all informed about the rule of law in their own country.
·Baseline: 15% (EU Survey on Justice, Rights and Values – event survey, 2022), 45% (Special Programme Eurobarometer, 2021).
·Results: 18% (event survey, Q3 2023), 45% (Special Programme Eurobarometer 2024).
·Overall assessment: It may prove challenging to achieve the desired decrease given that the results thus far indicate an increase of 3 percentage points in the event survey, and no change as regards the special programme Eurobarometer.
Indicator 5b: Awareness of the rule of law among the general public – in the EU
·Target 2027: Decrease (unspecified) in those who feel not very well informed or not at all informed about the rule of law in other EU Member States.
·Baseline: 39% (EU Survey on Justice, Rights and Values – event survey, 2022), 68% (Special Programme Eurobarometer, 2021).
·Results: 40% (event survey, Q3 2023), 68% (Special Programme Eurobarometer 2024).
·Overall assessment: It may prove challenging to achieve the desired decrease given that the results thus far indicate an increase of 1 percentage point in the event survey, and no change as regards the special programme Eurobarometer.
II. Progress towards specific objectives against baselines (per individual strand)
The programme has the following specific objectives, which correspond to the strands:
a)to protect and promote Union values (Union values strand);
b)to promote rights, non-discrimination and equality, including gender equality, and to advance gender mainstreaming and the mainstreaming of non-discrimination (equality, rights and gender equality strand);
c)to promote citizens’ engagement and participation in the democratic life of the Union and exchanges between citizens of different Member States, and to raise awareness of their common European history (citizens’ engagement and participation strand);
d)to fight violence, including gender-based violence (Daphne strand).
Indicator data are collected at the impact level on the change in event participants’ perception, behaviour, and practice for each specific objective via the EU Survey on Justice, Rights and Values (event survey).
3.3.3.1. Union values strand
Indicator 1: Change in participants’ behaviour
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they were likely to react differently when confronted with the topic / a similar situation / issue / case.
·Baseline: 86% (cumulative, all strands)
·Results: 84% (event survey, Q3 2023)
Indicator 2: Change in participants’ perception
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that their perception on the topic had changed.
·Baseline: 85% (cumulative, all strands)
·Results: 84% (event survey, Q3 2023)
Indicator 3: Change in participants’ practice
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they would use what they learned in their everyday life / work.
·Baseline: Baseline to be set 93% (cumulative, all strands)
·Results: 84% (event survey, Q3 2023)
3.3.3.2. Equality, rights and gender equality strand
Indicator 1: Change in participants’ behaviour
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they were likely to react differently when confronted with the topic / a similar situation / issue / case.
·Baseline: 86% (cumulative, all strands)
·Results: 80% (event survey, Q3 2023)
Indicator 2: Change in participants’ perception
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that their perception on the topic had changed.
·Baseline: 85% (cumulative, all strands)
·Results: 84% (event survey, Q3 2023)
Indicator 3: Change in participants’ practice
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they would use what they learned in their everyday life / work.
·Baseline: 93% (cumulative, all strands)
·Results: 92% (event survey, Q3 2023)
Indicator 4: Awareness of the General Data Protection Regulation
·Target 2027: Unspecified
·Baseline: 91% (event survey) and 67% (EB 2019)
·Results: 67% for women and 65% for men (event survey), 74% for men and 69% for women (Special Programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey is significantly below the baseline for both men and women (by 24 percentage points for women and 26 percentage points for men). However, the indicator for the special programme Eurobarometer is above the baseline for both men and women (an increase by 7 percentage points for men and 2 percentage points for women).
Indicator 5: Awareness of rights as an EU citizen
·Target 2027: Unspecified increase
·Baseline: 87% (event survey), 57% (Special Programme Eurobarometer)
·Results: 84% (event survey), 64% men and 58% women (Special Programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey against the baseline appears on track to being reached. The indicator for the special programme Eurobarometer against the baseline shows increases of 7 percentage points for men and 1 percentage point for women.
3.3.3.3. Citizens’ engagement and participation
Indicator 1: Change in participants’ behaviour
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they were likely to react differently when confronted with the topic / a similar situation / issue / case.
·Baseline: 86% (cumulative, all strands)
·Results: 81% (event survey, Q3 2023)
Indicator 2: Change in participants’ perception
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that their perception on the topic had changed.
·Baseline: 85% (cumulative, all strands)
·Results: 82% (event survey, Q3 2023)
Indicator 3: Change in participants’ practice
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they would use what they learned in their everyday life / work.
·Baseline: 93% (cumulative, all strands)
·Results: 87% (event survey, Q3 2023)
Indicator 4: Citizens’ engagement in civic activities, at local, national or European levels
·Target 2027: Unspecified
·Baseline: 59% (event survey), 25.8% (EB 2021)
·Results: 25% -78% on all items (event survey), 8%-21% for men and 9%-16% for women (special programme Eurobarometer 2024)
·Overall assessment: The indicator of the event survey was above the baseline for several items
Indicator 5: Citizens’ awareness of the common history
·Target 2027: Unspecified
·Baseline: 95% (event survey), 78% (EB 2021)
·Results: 91% (event survey), 79% for men and 75% for women (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey is below the baseline by 4 percentage points. As regards the indicator for the special programme Eurobarometer, it is above the baseline for men by 1 percentage point, but below the baseline by 3 percentage points for women.
Indicator 6: Citizens’ understanding of how the EU works
·Target 2027: Unspecified
·Baseline: 95% (event survey), 78% (EB 2021)
·Results: 91% (event survey), 79% for men and 75% for women (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey is below the baseline by 4 percentage points. As regards the indicator for the special programme Eurobarometer, it is above the baseline for men by 1 percentage point, but below the baseline by 3 percentage points for women.
Indicator 7: Citizens’ perception on democratic participation ‘my voice counts’ in the EU
·Target 2027: Unspecified
·Baseline: 68% (event survey), 42% (EB 2020)
·Results: 67% (event survey), 48% for men and 47% for women (EB 2024)
·Overall assessment: The indicator for the event survey is below the baseline by 1 percentage point. As regards the indicator for the Eurobarometer, it is above the baseline by 6 percentage points for men and 5 percentage points for women.
3.3.3.4. Daphne
Indicator 1: Change in participants’ behaviour
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they were likely to react differently when confronted with the topic / a similar situation / issue / case.
·Baseline: 86% (cumulative, all strands)
·Results: 87% (event survey, Q3 2023)
Indicator 2: Change in participants perception
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that their perception on the topic had changed.
·Baseline: 85% (cumulative, all strands)
·Results: 89% (event survey, Q3 2023)
Indicator 3: Change in participants practice
·Target 2027: Increase (unspecified) of those who reported ‘yes, definitely’ or ‘yes, to some extent’ that they would use what they learned in their everyday life / work.
·Baseline: 93% (cumulative, all strands)
·Results: 96% (event survey, Q3 2023)
Indicator 4 (also for the Equality, rights and gender equality strand): Percentage of EU citizens reporting having personally felt discriminated against or harassed within the previous 12 months in DG JUST area of competence
·Target 2027: Unspecified
·Baseline: 17% (EB 2019)
·Results: 18% of men and 24% of women who reported having personally felt discriminated against (EB 2023).
·Overall assessment: Both proportions are above the baseline of 17% set in 2019, by 1 percentage point for men, and 7 percentage points for women.
III. Progress towards specific objectives against baselines (all strands together)
Indicator 1: Awareness of the EU Charter of Fundamental Rights
·Target 2027: Unspecified
·Baseline: 90% (event survey), 42% (EB 2019)
·Results: 87% (event survey), 65% men and 60% women (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey against the baseline appears on track to being reached. The indicator for the special programme Eurobarometer against the baseline shows increases of 23 percentage points for men and 18 percentage point for women.
Indicator 2: Knowledge of EU legislation to promote and protect values
·Target 2027: Unspecified
·Baseline: 66% (event survey), 75% (EB 2021)
·Results: 79% (lowest value) – 88% (highest value) on all items listed (event survey), 70% - 79% men on all items listed, and 69%-74% women on all items listed (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey against the baseline has been surpassed on all items. The indicator for the special programme Eurobarometer against the baseline is below the baseline on the several items.
Indicator 3: Knowledge of awareness-raising actions to promote and protect EU values
·Target 2027: Unspecified
·Baseline: 31% (event survey), 43% (EB 2021)
·Results: 58% (lowest value) – 74% (highest value) on all items listed (event survey), 32% - 54% men on all items listed, and 31%-53% women on all items listed (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey against the baseline has been surpassed on all items. The indicator for the special programme Eurobarometer against the baseline is above the baseline on the several items.
Indicator 4: Knowledge of EU tools and initiatives to promote and protect EU values
·Target 2027: Unspecified
·Baseline: 39% (event survey), 48% (EB 2021)
·Results: 68% (lowest value) – 60% (highest value) on all items listed (event survey), 51% - 31% men on all items listed, and 49%-27% women on all items listed (special programme Eurobarometer 2024)
·Overall assessment: The indicator for the event survey against the baseline has been surpassed on all items. The indicator for the special programme Eurobarometer against the baseline is above the baseline on the several items.
Indicator 5: Knowledge of EU funding to promote and protect EU values
·Target 2027: Unspecified
·Baseline: 64% (event survey), 31% (EB 2021)
·Results: 62% who replied that they had heard about the programme and knew or did not know what it was (event survey), 37% for men and 35% for women (special programme Eurobarometer 2024).
·Overall assessment: The indicator of the event survey is 2 percentage points below the baseline. The indicator for the special programme Eurobarometer is above the baseline – by 6 percentage points for men and 4 percentage points for women.
Annex III. Evaluation matrix
The evaluation design relied on four evaluation matrices (see tables below) built along the five Better Regulation Guidelines’ evaluation criteria that guided the evaluation process. Based on the programmes’ intervention logics and the supporting study technical specifications, the matrices feature the key research questions, indicators and related judgement criteria. The data sources are included in the table.
Additional to the three evaluation matrices designed to assess each programme, a fourth cross-cutting analytical framework was developed to evaluate the synergies between the different programmes and programming strands. With the additional framework it was possible to explore the added value of bringing all strands within the architecture of the CERV programme, and supported the identification of lessons learned. The analytical framework was developed and structured as an evaluation matrix with evaluation questions, sub-questions and judgement criteria under each evaluation criteria. This structured the analysis and ensured its comprehensiveness. The indicators informing the evaluation questions consisted of the findings from the evaluation of the REC, EfC and CERV programmes (based on the answers to the respective programmes’ evaluations’ questions in line with the three separate matrices drafted for each of the programmes). Individual responses to the cross-cutting analytical framework were not drafted to avoid duplication but directly informed the overall conclusion of the evaluation and supported the identification of lessons learned across all evaluation criteria as relevant.
Evaluation questions matrix of the REC programme
|
Evaluation question
|
Sub-question
|
Judgement criteria (There is evidence that…)
|
Example of indicators
|
Sources
|
|
Effectiveness
|
|
1.To what extent did the REC programme contribute to promoting and protecting equality and fundamental rights as enshrined in EU primary law?
|
1.1How successful has the REC programme been in achieving its objectives?
|
The SOs were achieved and contributed equally to the achievement of the programme’s general objective of effectively promoting and protecting equality and fundamental rights.
It is possible to identify reasons why the specific objectives have been achieved to a greater or lesser extent, thus contributing to a greater or less extent to fulfilment of the general objective.
|
Evidence from EQ1 on contribution of actions to achieving specific objectives and on achievement of specific objectives
|
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
2.What are the most relevant long-lasting effects of the REC programme, and why?
|
2.1 What are the key long-term results and impacts, and for what reasons can these effects be considered long-lasting?
2.2. What are the actions funded under the programme that produced the long-term effects?
|
EU Citizens’ rights have been effectively promoted, protected and implemented (as per general objective) through achievement of the specific objectives (as per EQ1), the effects are still discernible and are likely to continue, i.e.
-implementation of the principle of non-discrimination on grounds of sex, racial or ethnic origin, religion or belief, disability, age or sexual orientation has been promoted;
-racism, xenophobia, homophobic and other forms of intolerance have been combated;
-the rights of persons with disabilities have been promoted and protected;
-equality between women and men has been promoted and gender mainstreaming has been advanced;
-all forms of violence against children, young people and women, violence against other groups at risk have been prevented and combated;
-the rights of the children have been promoted and protected;
-a contribution has been made to ensuring the highest level of protection of privacy and personal data;
-the exercise of European citizenship rights has been promoted and enhanced
-the rights of individuals and entrepreneurs as consumers have been enforced.
|
Improvement in the level of knowledge of Union law and policies and, where applicable, of rights, values and principles underpinning the Union
Achievement of the target that all Member States set up structural coordination mechanisms with all stakeholders, including Roma, on the implementation of the National Roma Integration Strategies
Achievement of targets / Rising and sustained improvement in programme indicators listed in DG Annual Activity Reports and Statements
Appropriate geographical balance throughout the life of the programme
Stakeholders’ perceptions that long-lasting results have been achieved, i.e. the understanding of rights, the protection of rights and the prevention of various forms of harm and abuse have improved, and the achievements have been sustained or have continued since the end of the programme
|
Scoping interviews
Open public consultation
Surveys
Interviews with beneficiaries
Interviews with beneficiaries
|
|
3.Were there any unexpected or unintended factors that occurred in the implementation period, and which drove or hindered progress?
|
|
The achievement of the REC objectives was positively or negatively affected by external factors, such as:
External factors international, national political and economic contexts (inter alia policies which restricted or opened up the civic space, the COVID-19 pandemic and its economic consequences and the Russia’s war of aggression against Ukraine) were barriers to achieving the specific objectives / created a more favourable environment than expected.
|
Evidence of any unexpected or unintended external factors which drove or hindered progress in relation to the implementation of the programme
Stakeholder perception of any unexpected or unintended external factors that occurred, and which drove or hindered progress
|
Scoping interviews
Desk research
Open public consultation
Targeted survey of beneficiaries
Interviews with beneficiaries
|
|
4.How evenly were the effects distributed across the different target groups and participating countries?
|
4.1 How can the variation in the programme’s effects across target groups and Member States be explained?
|
The dissemination of information, applications and awards were distributed evenly across the different participating countries.
The dissemination of information, applications and awards were distributed evenly across the different target groups.
Where there are variations between target groups and/or countries, it is possible to identify (internal and external) reasons.
|
Number of applications and grant awards by country, by year, action v operating grants, correlated with share of funding by type of beneficiary, target groups, specific objective
Number of procurement contracts awarded by year, specific objective, and type and results of those procurement contracts
Success rates by specific objective (and the groups they target) and by country
Stakeholder perceptions of reasons for country-by-country variations legal context, developments in civic space
|
Scoping interviews
Desk research
Open public consultation
Interviews with beneficiaries
|
|
Efficiency
|
|
5.Have the effects of the programme been achieved at initially expected costs or were these costs different?
|
5.1 What could explain the differences (if any)?
|
The programme budget was fully committed and paid out.
The actions under the programme were completed at the same or a lower cost than expected.
Underspending and/or overspending applied to certain types of action, certain specific objectives, or certain periods of implementation.
There were factors in the internal and external environment which resulted in costs being different from what had been expected.
|
programme budget amount planned by year and budget outturn (by year, by specific objective and type of action)
Budget data from actions clustered by type of action and if relevant by Member State
Stakeholder perception of planning and management at programme and action/participating organisation level, support received from the Commission, including the possibility for reallocating funds
Stakeholder perceptions of reasons for variability
|
Scoping interviews
Desk research, including analysis of indicators
Open public consultation
Surveys
Interviews with beneficiaries
|
|
6.To what extent were the administrative costs of the programme justified, given the effects it achieved?
|
|
Processes for application are efficient (timely, robust, transparent):
-Deadlines for submission were reasonable;
-evaluation processes were robust, incl. transparency selection process;
-the length of time between submission and award was proportionate;
-the time between grant award and grant signature was proportionate.
Application process did not impose a disproportionate burden.
Monitoring and reporting requirements did not impose a disproportionate burden.
The administrative burden was proportionate for all sizes of projects and types of organisation applying.
|
Qualitative data on management and control costs (e.g. internal audit, European Court of Auditors reports and performance audits)
Data/ stakeholder views on submission periods, time to inform, time to grant, time to pay and trends
Stakeholder perceptions of management and control costs, time periods and administrative costs (by size and type of organisation) for applicants of submitting applications and reporting and monitoring
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
7.Could the programme have been implemented in a more efficient way?
|
7.1 To what extent were any previously proposed simplification measures applied?
|
There were areas in programme (project management, application and selection process, co-financing rate) where there are opportunities for efficiency gains (i.e. outputs and outcomes could have been achieved at a lower cost).
Factors adversely or positively influencing efficiency could have been tackled more appropriately / expeditiously.
Better communication would have resulted in higher quality applications and better geographic balance.
The introduction in 2014 of risk assessment for final payments reduced the administrative burden on the Commission and beneficiaries and overall reduced payment delays.
Migration to a more paperless system part-way through the programme reduced costs to the greatest extent possible.
|
Data from internal audits, European Court of Auditor Reports, performance audits, part 1 of this evaluation on potential for efficiency gains
Stakeholder perceptions of efficiency gains from simplification measures
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
Relevance
|
|
8.How well did the original objectives of the programme still correspond to the needs within the EU?
|
|
JC 8.1: the specific objectives of the REC programme addressed the needs of the EU.
|
Application levels relative to the themes of calls for proposals by specific objective, year and country
Number and percentage of persons in a target group reached by the awareness-raising activities funded by the programme.
Number of stakeholders participating in, inter alia, training activities, exchanges, study visits, workshops and seminars funded by the programme by topic
Improvement in the level of knowledge of Union law and policies and, where applicable, of rights, values and principles underpinning the Union, in the groups participating in activities funded by the programme compared with the entire target group Stakeholder perceptions that the needs initially identified persist (and the extent to which it is true of all)
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
|
|
JC 8.2: The REC programme evolved to adapt to the emergence of new needs.
|
Stakeholder perceptions of new needs and the extent to which the programme adapted to them
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
9.How relevant was the programme to its beneficiaries?
|
9.1 What are the key lessons learned?
|
JC 9.1: The types of action, e.g. improving the knowledge base, sharing best practices through mutual learning, capacity building and structural support for specific organisational structures, and the priorities implemented under the programme were appropriate for the needs of the final beneficiaries.
JC 9.2: The beneficiaries of funding did not adapt their activities and priorities disproportionately in order to obtain the funding.
JC 9.3: The mix of beneficiaries in consortia and partnerships was appropriate.
JC 9.4: The ways in which the programme was not relevant to its final beneficiaries provide lessons to be learned for the future.
|
Data from satisfaction surveys of participants and/or groups reached by activities under the programme
Stakeholder perceptions as to whether the types of action funded by the programme were those best suited and were delivered in a way that best suited the organisations and the needs they were seeking to meet and the target groups they address, and the extent to which this was the case irrespective of the size or the public or non-governmental institutional nature of those beneficiaries
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
Coherence
|
|
10.To what extent was the programme coherent with wider EU policies and priorities?
|
10.1 Which synergies and/or complementarities (if any) could be identified between the programme and other EU instruments?
|
The programme contributed to the priorities of the European Commission in the period 2014-2020, notably the social priorities of the Europe 2020 agenda, and to the green and digital transitions which were priorities from 2020
The REC programme contributed to:
·the EU justice agenda for 2020;
·the 2016-2020 strategic engagement for gender equality;
·the 2010-2020 European disability strategy;
·the list of actions by the Commission to advance LGBTI equality;
·the EU framework for national Roma integration strategies.
The European Commission/the programme developed synergies and was complementary with other EU funds with similar objectives, such as the Justice programme, the Consumer programme, EASI, and other programmes in the areas of employment and social affairs (ESF and ERDF); home affairs (ESF-BV and ESF-Police); health and consumer protection; education, training, youth and sport; the information society; enlargement (notably IPA II)
The European Commission/the programme developed synergies and was complementary with the European Institute for Gender Equality (EIGE) and the Agency for Fundamental Rights (FRA), and other national and international actors in the areas covered by the programme.
|
Documentary evidence of planned synergies and complementarities between the REC programme objectives and other EU policies and priorities.
Stakeholder perception of coherence between the REC programme objectives and other EU policies and priorities
|
Scoping interviews
Desk research
Open public consultation
Surveys
Interviews with beneficiaries
|
|
EU Added value
|
|
11.Could the identified impacts have been achieved without the programme?
|
|
The following added value impacts could not have been achieved without the programme:
·consistent and coherent implementation of Union law;
·wide public awareness about the rights deriving from Union law;
·mutual trust among participating countries and improved cross-border cooperation;
·developing and disseminating best practices;
·creating standards, practical tools and solutions that address EU-wide challenges.
|
Number of cases, activities, and outputs of cross-border cooperation
Evidence of implementation of EU law, dissemination of best practice (e.g. peer review reports), standard and tools, best practice in communication/visibility
Stakeholder perception that only a programme of this kind can deliver such impacts evenly across all Member States, either in the absolute or in order to fill gaps in what is available at national level
|
Desk research
Surveys
Open public consultation
Interviews with beneficiaries
|
|
12.Considering that the initial problem and its causes varied across the national levels, to what extent did the programme help establish a level playing field?
|
12.1 Which were the most significant transnational/cross-border aspects the programme tackled? Could these be quantified?
|
The programme reduced gaps across Member States in the extent of the problems identified in the Impact Assessment preceding the REC existed across the Member States, i.e.
·insufficient or inconsistent implementation of certain rights across the Union, and
·lack of awareness about certain Union legislation both by citizens and public authorities.
The programme helped rectify imbalances in the civic space.
It is possible to identify certain aspects or themes of transnational/cross-border cooperation that received particular emphasis.
|
Stakeholders’ perception that the programme has helped achieving a level playing field.
Evidence of best practices, standards, practical tools and solutions that address EU-wide challenges
Evidence of funding for innovation and transnational actions
Evidence of increased mutual trust among participating countries and cross-border cooperation
|
Desk research
Surveys
Open public consultation
Interviews with beneficiaries
|
Evaluation questions matrix of the EfC programme
|
Evaluation question
|
Sub-question
|
Judgement criteria (There is evidence that …)
|
Example of indicators
|
Sources
|
|
Effectiveness
|
|
1.What are the most relevant long-lasting effects of the EfC programme, and why?
|
1.1To what extent can be the actions funded under the programme be considered sustainable?
1.2To what extent have the effects of the EfC continued in the period 2021-2023
|
Project activities and/or cross-border collaboration and/or networks continued after the end of the programme either within CERV or in other ways.
Over the life of the programme, there was an increase, which had since continued or been maintained in:
oa sense of EU citizenship (belonging to the EU);
oincreased participation in civic life;
oincreased participation in democratic life.
It is possible to discern success factors.
|
·Data on recurrent beneficiaries
·Examples of collaboration between beneficiaries in similar events/projects after initial event/activities/projects
·Examples of transfer of good practice
·Data from peer reviews
·Quality of the project applications and the degree to which the results of selected projects can be further used/transferred
·Number and quality of policy initiatives following-up on activities supported by the programme at the local or European level
·Data from Eurobarometer
Stakeholder perceptions of long-term results and the reasons for that
Participants’ assessment of the activities in which they participated and of their (expected) sustainability (if available from Surveys carried out at the time)
|
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Desk research
Public consultation
|
|
|
|
|
|
|
|
2.How successful has the EfC programme been in achieving its objectives?
|
2.1To what extent has the programme achieved its objectives and what were key success factors?
|
It is possible to make a quantitative or qualitative assessment of the extent to which the programme contributed to its objectives (raising awareness of remembrance, common history and values, and democratic and civic participation).
It is possible to differentiate between the strands in the quantitative or qualitative assessment of the extent to which the programme contributed to its objectives.
The horizontal aspect of the programme led to valorisation and transferability of results with long-term impact.
It is possible to identify (internal and external) factors that contributed particularly strongly to the achievement of the objectives.
|
Mapping of the effects/results achieved and assessment in how far these correspond to the initial objectives.
Performance against targets in numbers of persons reached directly or indirectly by objective/strand
Performance against targets in number of participating organisations/towns
Performance against targets in number of transnational partnerships and diversity of participants in those partnerships
The number of first-time applicants by Member State
Perceptions of the stakeholders on the effectiveness of the programme overall and by strand, and the impact of funded projects and success factors
Target groups’ perception that they feel more aware of the EU after being involved in projects, and that they understand better the Union history and diversity and/or want to be more involved in civic society (if data were collected at the time of participation).
Number and quality of policy initiatives following-up on activities supported by the programme at the local or European level
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Public consultation
|
|
3.Were there any unexpected or unintended effects that occurred, and which drove or hindered progress? What can explain these effects?
|
|
It is possible to identify factors that led to some actions exceeding or falling short of expectations / failing to achieve their objectives in unforeseen / unforeseeable ways.
There are patterns to explain some factors, but others are the result of specific national contexts.
There are lessons to be learned from the implications of these factors for progress.
|
Data from final project technical reports and DG JUST/EACEA activity reports on unexpected or unintended effects and the drivers for these
Stakeholders’ perception of unexpected or unintended effects, and the drivers
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Public consultation
|
|
4.How evenly were the effects distributed across the different target groups and participating countries?
|
4.1 How can the variation in the programme’s effects across target groups and Member States be explained?
|
There is evidence of the extent to which the effects were distributed evenly across the different target groups and participating countries.
The findings of the mid-term evaluation that there were only minor variations, except in the case of the geographical balance of town twinning, are confirmed.
Dissemination of information about the programme was even across the Member States and appropriately targeted to groups under-represented in applications or award.
Where there are variations between target groups and/or countries, it is possible to identify (internal and external) reasons.
|
Data on geographical coverage of the activities relative to the number of people directly and indirectly involved correlated with target groups data (where available) and total population, and by strand and type of action
Eurobarometer and other Surveys data (where available) on increases in in the level of civic participation and participation in democratic life (including elections to the European Parliament) across all the target groups.
Number of applications by country and by year, and success rates by country
Stakeholder perceptions of reasons for differences across target groups and Member States
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Public consultation
|
|
5.How effectively did the structure of the programme and the types of supported actions – particularly, town-twinning and networks of towns, remembrance activities and civil society projects – contribute to achieving the general and specific objectives?
|
|
The structure of the programme, i.e. the structural separation between town twinning and networks or towns, and the existence of separate strands for remembrance and civil society activities was helpful in achieving distinct, distinguishable, at least qualitatively measurable, and logically separate contributions to achieving the general and specific objectives.
Compared to the previous programming period, actions/measures became clearer and more focused.
The finding of the mid-term evaluation that Town Twinning and Networks of Towns should be merged is not confirmed.
|
Stakeholders’ perceptions that the structure of the programme was well understood and clearly distinguished between the objectives and types of actions, so that it was possible to differentiate between the relative contributions to each objective and this demarcation was a success factor
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Public consultation
|
|
Efficiency
|
|
6.To what extent has the programme been cost-effective?
|
|
The outputs were delivered with or at less than the allocated budget.
The resources and costs allocated to the actions were sufficient, available on time and reasonable when compared to the outputs.
The cost structures across funded projects were broadly comparable.
The costs and benefits were comparable to similar projects funded from other sources.
There were no issues of underspending or overspending.
The advantages of funding recurring projects or continuation of projects outweigh the disadvantages of funding a wider group of beneficiaries.
There were no significant barriers to applying for funding relating to language, capacity, relevance, awareness of the programme or other factors.
|
Project cost data, including average costs per person per comparable type of activity (subject to suitable cost data being available)
Stakeholder explanations for discrepancies (if any) between the budget allocated to the programme and the budget used (e.g. insufficient quality of proposals or absorption capacity)
Stakeholder explanations for differences in cost structure that go beyond different price levels in different countries, and for under- or overspending
Stakeholder perceptions of the benefits of funding recurring or continuing projects
Stakeholder perceptions that the programme has been cost-effective
Stakeholder perceptions that there were no significant barriers to applying
|
Desk research
Interviews with beneficiaries
|
|
7.How efficient was the governance of the programme?
|
|
The European Commission had sufficient human and financial resources (inputs) to achieve programme management objectives.
The management and control system established for the implementation of the programme was appropriate for delivery of the outputs and achieving the outcomes.
The programme Committee contributed to effective governance at a reasonable cost.
The Civil Dialogue achieved its objectives of encouraging exchanges of experiences and good practices, contributing to the dissemination of the programme’s results, and monitoring and discussing discusses policy developments in related fields of the programme at a reasonable cost.
|
Minutes of the meetings of the programme Committee and the Civil Dialogue Group
EC stakeholder perception that they had adequate financial and human resources available to manage the programme efficient
Stakeholder perceptions of the adequacy of the resources for programme management and of the management and control system
Stakeholder perceptions of good planning and organisation, including financial planning
Stakeholder perceptions of the value of the contributions of the programme Committee
Stakeholder perceptions that the Civil Dialogue achieved its objectives and did so efficiently.
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Targeted survey of NCPs
Public consultation
|
|
8.Could the programme have been implemented in a more efficient way?
|
8.1 To what extent were any previously proposed simplification measures applied?
|
There were areas in programme and project management where outputs and outcomes could have been achieved at a lower cost.
Factors adversely or positively influencing efficiency could have been tackled more appropriately / expeditiously.
Better communication would have resulted in higher quality applications and better geographic balance.
The administrative burden of applying and reporting was appropriate.
The time taken to evaluate project applications and to sign grant agreements was appropriate.
The use of unit costs for European Remembrance, Networks of Towns and Civil Society Projects achieved the intended simplification.
Lump sums for preparatory activities for European Remembrance and Civil Society Projects and for town twinning achieved the intended simplification.
|
Indicators on time taken to evaluate and award projects, and sign grant agreements
Evidence of good planning and organisation, especially financial planning
Evidence of anticipation of any over or underspending in order to take prompt corrective steps
Stakeholder perceptions of the efficiency of programme management and of the level of administrative burden
Stakeholder perceptions of the use in specific contexts of lump sums and unit costs
Stakeholder perceptions of the appropriateness of the level of the unit costs
Stakeholder recommendations for further improvements
|
Answer to EQ7
|
|
Relevance
|
|
9.How did the objectives of the programme correspond to wider EU policy goals and priorities?
|
|
The objectives of the programme were appropriate to achieving the wider policy goals identified in Regulation 390/2014, e.g.
-exercise of the right to participate in the democratic life of the Union and the Union’s institutions;
-open, transparent and regular dialogue with representative associations and civil society;
-‘Europe 2020 – A strategy for smart, sustainable and inclusive growth’, aiming to deliver growth, employment, productivity and social cohesion.
|
Perception of the Union and its institutions by the beneficiaries
Perception of stakeholders that:
-the programme contributed to:
-greater participation in democratic and civic life
-more dialogue with civil society
-to social cohesion
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Public consultation
|
|
10.How relevant was the programme to its final beneficiaries?
|
10.1 What are the key lessons learned?
|
The types of action, e.g. improving the knowledge base, sharing best practices through mutual learning, capacity building and structural support for specific organisational structures, and the thematic priorities implemented under the programme were appropriate for the needs of the final beneficiaries.
The beneficiaries of funding did not adapt their activities and priorities disproportionately in order to obtain the funding.
The mix of beneficiaries in consortia and partnerships was appropriate.
The ways in which the programme was not relevant to its final beneficiaries provide lessons to be learned for the future.
|
Number of applications received under each strand
Types of activities funded by year
Stakeholders’ perception that their needs have been met and they did not have to adapt their objectives, policies or project in order to fit the priorities of the programme
Stakeholders’ perceptions that all consortia and partnerships were well structured, i.e. based on existing relationship, active contributions by all partners, and were not put together solely for presenting a proposal
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Public consultation
Focus group with Programme Committee Members
|
|
Coherence
|
|
11.To what extent was the programme coherent with wider EU policies and priorities?
|
11.1. Which synergies and/or complementarities (if any) could be identified between the programme and other EU instruments?
|
The programme contributed to the priorities of the European Commission in the period 2014-2020, notably the social priorities of the Europe 2020 agenda, and to the green and digital transitions which were priorities from 2020.
The programme developed synergies with policies in the areas of especially education, vocational training and youth, sport, culture and the audiovisual sector, fundamental rights and freedoms, social inclusion, gender equality, combating discrimination, research and innovation, information society, enlargement and the external action of the Union, and via NCPs with implementation mechanisms such as Creative Europe Desks, Erasmus+ National Agencies, Europe Direct Information Centres, Eurodesk information centres etc.
Direct overlap between the EfC and other funding programmes, notably the Rights Equality and Citizenship programme, Erasmus+ and Creative Europe’s Culture sub-programme in terms of content, objectives and target groups was minimal.
|
Evidence of synergies and complementarities developed by design between the EfC programme objectives and other EU policies and priorities
Evidence that overlaps and duplication with other programmes, notably the Rights Equality and Citizenship programme, Erasmus+ and Creative Europe’s Culture sub-programme was avoided by design
Stakeholder perception of coherence between the programme objectives and other EU policies and priorities
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Focus group with Programme Committee Members
Public consultation
|
|
EU Added value
|
|
12.Could the identified impacts have been achieved without the programme?
|
|
It would not have been possible to bring Europe closer to its citizens, enhance the sense of belonging or understanding of the Union, its history and diversity or increase participation in civic and democratic life to the same extent without the actions under the programme.
There were no national or transnational programmes that could have fulfilled the role of the EfC.
|
Stakeholder perception of the likelihood of the impact of the programme having been achieved (or at least not to the same extent) without the Programme (or with less funding)
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Public consultation
|
|
13.If the initial problem and its causes varied across the national levels, did the Programme help establish a level playing field?
|
13.1 Which were the most significant transnational/cross-border aspects the Programme tackled? Could these be quantified?
|
The Programme reduced gaps across Member States in the extent of the problems identified in the Impact Assessment preceding the EfC existed across the Member States, i.e.
-civil society’s capacity to participate in the EU policy making process;
-opportunities for individual citizens to participate in debates and discussions on EU-related issues.
The Programme helped rectify imbalances in the civic space.
It is possible to identify certain aspects or themes of transnational/cross-border cooperation that received particular emphasis.
|
Qualitative date from final technical report on civil society capacity building
Geographic balance of numbers of people directly and indirectly involved in EfC programmes and numbers of projects
Data available (either existing or collected as part of this study) to assess qualitatively or quantitatively the most significant transnational/cross-border aspect of the Programme
Stakeholder perceptions based where possible on an assessment of the quantitative and qualitative data on the Programme results that the Programme has helped achieve a level playing field and counter a shrinking civic space
Stakeholder perception that mutual trust among participating countries and cross-border cooperation have increased as a result of the Programme
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Public consultation
|
Evaluation questions matrix of the CERV programme
|
Evaluation question
|
Sub-question
|
Judgement criteria (There is evidence that …)
|
Example of indicators
|
Sources
|
|
Effectiveness
|
|
1.How successful has the programme been in progressing towards its objectives?
|
1.1 How likely is it that the projects funded by the programme so far promote EU values and contribute to supporting CSOs working in the field?
|
The projects, and their proposed activities/outputs, are aligned with the type of action envisaged by the programme across all strands, i.e. they fall within one of the 12 categories listed in Annex I of the Regulation.
The programme is on track to meet its indicator targets.
Promotion of EU values and support for CSOs working in the field are cross-cutting themes in calls across all strands.
Awarded projects are
·in line with EU policy priorities;
·address a real need in the field;
·based on well-established methodologies;
·(where relevant) are based on strong partnerships established in advance;
·have a strong EU dimension and offer EU added value;
·plan realistic and sustainable results.
|
Statistical trends and data analysis insights from an aggregate overview by strand of the quantitative and qualitative indicators on outputs and results collected under EQ2
Analysis of evaluation of applications and successful projects
Stakeholder perceptions of the trends and insights from the aggregate of quantitative indicators is an indication of success in achieving the objectives by strand and from an overall programme perspective
Perceptions of relevant stakeholders from their own work in the field of the progress being made and the likelihood of the projects funded so far contributing to supporting CSOs
Empirical evidence from studies, conference presentations etc. of activities that contributed to achieve the general objectives:
·ensuring protection and promotion of rights and values,
·sustaining and further developing open, rights-based, democratic, equal and inclusive societies which are based on the rule of law
·supporting civil society and encouraging people’s democratic, civic and social participation
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Case study 4
|
|
2.What have the quantitative and qualitative effects of the programme been? – answer provided under EQ1
|
2.1 Are there any significant differences compared to what was expected when designing the programme?
2.2 What are the effects of the novelties introduced?
2.3 How likely is it that the actions implemented so far will generate long-term effects?
|
There is data available to measure the quantitative outputs of the first three years of the programme.
Data on outputs suggest that milestones for 2024 are likely to be achieved without any significant deviations.
It is likely the outcomes envisaged in the Intervention Logics will be achieved.
Planning has been facilitated for both beneficiaries and DG JUST/EACEA by having multiannual work programmes.
Lump sums and unit costs are making it easier for beneficiaries to concentrate on achieving objectives.
The indicators are well suited to measuring effects.
Projects are more results-oriented than in the past.
The cascade grants system is making EU funding more available to smaller/grassroots organisations.
Projects and consortia are operating along a continuum designed to produce long-term impacts.
|
Data on outputs of the programme and of projects, i.e. numbers of people, organisation and action types taking into account success rates and geographical balance, and funding distribution by type of beneficiary (where available: disaggregated by sex, age, disability or other axes of differentiation relevant for the purpose of the evaluation, such as sexual orientation, race and ethnicity, religion).
Qualitative data on outcomes from indicators on changes in awareness etc. and behaviour (where available: disaggregated by sex, age, disability or other axes of differentiation relevant for the purpose of the evaluation, such as sexual orientation, race and ethnicity, religion).
Percentage of projects that are a continuation of previously funded projects / Percentage of beneficiaries that are recurrent beneficiaries and/or in recurring consortia
Data on whether indicators for which there are milestones in performance statements are on track
Empirical examples of actions which are likely to generate long-term effects
Empirical examples of high-quality dissemination and communication plans
Stakeholders’ perception that effects have been achieved and that projects are more results-oriented, including the perception of whether effects are likely to be long term
|
Desk research
Targeted survey of beneficiaries
Focus group with Programme Committee Members
Interviews with beneficiaries
Survey of NCPs
Focus group with NCPs
Deliberative workshops
Case study 2
|
|
3.What external factors have affected progress towards the objectives and how are they linked to the programme?
|
|
Factors such as COVID-19, the Russia’s war of aggression against Ukraine, Brexit, specific national contexts (changes of government, the halt to signature of grant agreements with Hungary, introduction of Member State policies inimical to EU values, other specific national contexts) have affected progress towards all or some objectives.
|
Assessments in documentary evidence (Commission reports and other studies) of external factors which drove or hindered progress in relation to the implementation of the programme
Stakeholder perceptions of any external factors that occurred, and which drove /are driving or hindered / are hindering progress
|
Desk research
Targeted survey of beneficiaries
Interviews with beneficiaries
|
|
4.Compared to its predecessors, how effectively has the programme targeted policy objectives and beneficiaries?
|
4.1 Which aspects were better addressed?
4.2 How effectively has it addressed programme management and economies of scale?
|
The programme and the themes it is targeting have a clearer link with policy objectives than its predecessors.
The programme and the way in which it is being implemented is more targeted than its predecessors.
It is clear which beneficiary groups are being targeted by different strands and themes
Some aspects of the targeting are more successful than others.
The economies of scale resulting from the merger of the EfC and REC are identifiable are being optimised and have led to improved targeting.
|
Stakeholder perceptions of areas where the effectiveness of targeting has improved and this is the result of changes resulting from better management after merger of EfC and REC and has been derived from economies of scale
|
Mid-term and ex post evaluations of REC and EfC
Desk research
Survey of beneficiaries
Interviews with beneficiaries
Case study 2
|
|
5.How effective have the communication activities been in informing stakeholders about and engaging them in the programme and in increasing the programme’s visibility?
|
5.1 How can the communication activities be adapted to increase stakeholder engagement and the programme’s visibility?
|
The communication activities of DG JUST, EACEA and the Contact Points have been successful in informing stakeholders about the programme.
Delays in appointing Contact Points in some Member States / the absence of Contact Points in some Member State has not been detrimental to the programme’s visibility.
There are improvements to be made in the means of communication and stakeholder engagement, e.g. in the exchanges & dissemination of good practices amongst beneficiaries and potential applicants; in networking among Contact Points; in the type and distribution of communication tools, and the use of social media.
|
Correlation between first-time applicants and first-time transnational partnerships and communication activities, e.g. central info sessions, social media, work of NCPs
Stakeholder perception that the clarity or intensity of communication, the type of communication and media is appropriate / needs to be adapted
|
Desk research
Targeted survey of beneficiaries
Targeted survey of NCPs
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
|
|
6.To what extent have the actions funded under the programme so far responded to the needs of the target stakeholders?
|
6.1 To what extent are those actions sustainable?
|
Target stakeholders identify with the needs the actions funded under the programme are designed to address.
Target stakeholders have not needed to adapt what they see as relevant priorities in order to fit the requirements of the programme.
|
Number of projects that are not in line with the core business of the beneficiary out of the total number of funded projects (business opportunity)
Absorption capacity by policy priority/strand, i.e. the ratios between number of applicants, number of successful applicants, quality of applications (as reflected in average scores) and funding available per priority/strand
Stakeholder perceptions that the types of action funded are appropriate for the objectives of the target stakeholders
|
Targeted survey of beneficiaries
Interviews with beneficiaries
Case study 4
Focus group with NCPs
|
|
7.How evenly were the effects distributed across the different target groups and participating countries?
|
7.1 How can the variation in the programme’s effects across target groups and Member States be explained?
|
There is evidence of the extent to which the effects were distributed evenly across the different target groups and participating countries.
Dissemination of information about the programme was even across the Member States and appropriately targeted to groups under-represented in applications or award.
Where there are variations between the effects by target groups and/or countries, it is possible to identify (internal and external) reasons.
|
Results of the statistical analysis under EQ2
Documentary evidence from reports and studies of reasons for variability
Stakeholder perceptions of reasons for variability in the actions of DG JUST and EACEA and in the national and international environment
|
Desk research
Targeted survey of beneficiaries
Targeted survey of NCPs
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
|
|
8.How effective has the programme been in terms of promoting gender equality?
|
8.1 What is the (prospective) impact of the programme on gender equality?
8.2 How effectively has gender equality been mainstreamed in the programme design and implementation?
|
The introduction of the requirement to include a gender perspective in all proposals has been well complied with across all strands and themes.
There are aspects of gender equality where the programme is expected to have a more significant impact.
There are no unintended negative effects on gender equality.
The gender score in EC budget measurement of horizontal priorities compares favourably with equivalent funding programmes.
|
Disaggregated data under EQ2, including on gender identity and diversity where available
Degree to which the programme has promoted implementation of the gender equality strategy
Data on gender mainstreaming in projects
Gender balance in consultation of experts and stakeholders
Annual gender scores in EU budget management
Stakeholder perceptions of the extent to which the programme is advancing gender equality, and strengths and weakness of its gender equality promotion
Contribution to implementation of EC gender equality strategy 2020-2025 & equal representation of women, men in EP post-2024; design, development, monitoring of national, regional, local = representation policies; increased capacity of CSO’s at all levels, notably women’s organisations, and media; cooperation, networking, information exchange across EU of information between Member State governments, local, regional authorities, associations, equality bodies, CSOs (notably women’s), media, academics.
Awareness raising & training of professionals; changed attitudes & behaviours; identification with progressive gender portrayal & positive messaging.
|
Case study 1
Desk research
|
|
Efficiency
|
|
9.What are the costs and benefits of the programme for the different stakeholders?
|
|
There are quantitative or qualitative data on the costs and benefits for different stakeholders.
Average costs of actions are similar for groups of countries with similar cost structures.
There are identifiable reasons and lessons to be learned from variability in costs and benefits across stakeholders.
|
Budget amount planned by year and by specific objective
Share of funding by type of beneficiary
Level of funding requested by applicants and granted in relation to each specific objective
Average cost of an awareness-raising, information and dissemination activity (by country and specific objective)
Average cost of a mutual learning and exchange of good practices activity (by country and specific objective)
Average cost of a training activity (by country and specific objective)
Average cost of one person trained (by country and specific objective)
Average cost of one person/day/training (by country and specific objective)
Stakeholder perception that the programme has been cost-effective for each type of stakeholder
Stakeholder perceptions of reasons for cost variations
|
Desk research
Targeted survey of beneficiaries
Targeted survey of unsuccessful applicants
Survey on costs (beneficiaries)
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
|
|
10.To what extent have internal or external factors influenced the efficiency to generate results?
|
|
Internal factors likely to affect costs and benefits, e.g. delays in appointing Contact Points (which may also be the result of external factors), the halt to signature of grant agreements with Hungary, start-up problems with the new IT platform.
External factors which may be affecting costs and benefits, e.g. digitalisation, COVID-19, the Russia’s war of aggression against Ukraine, Brexit, specific national contexts (changes of government, introduction of Member State policies inimical to EU values, other specific national contexts) have affected progress towards all or some objectives.
|
Documentary evidence of factors influencing efficiency
Stakeholder perceptions of factors influencing efficiency
|
Desk research
Targeted survey of beneficiaries
Interviews with beneficiaries
Deliberative workshops
Focus group with NCPs
Focus group with Programme Committee Members
|
|
11.Have any inefficiencies been identified?
|
|
There is scope for reducing costs without any loss of benefit and for increasing benefits at the same cost.
The opportunities of digitalisation are not being optimised at programme or project level.
|
Documentary evidence from internal audit reports, performance audits and European Court of Auditor reports
Indicators on ICT tools (see EQ2)
Stakeholder perceptions of potential for efficiency gains:
- in the use of human and/or financial resources
- optimising the use of ICT tools
|
Desk research
Targeted survey of beneficiaries
Targeted survey of unsuccessful applicants
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
|
|
12.How timely and efficient is the programme’s administrative process, including for reporting and monitoring?
|
|
The administrative burden is proportionate for all size of project and organisation.
Deadlines for submission are reasonable, evaluation processes are robust, the length of time between submission and award is proportionate, the time between grant award and grant signature is proportionate.
Monitoring and reporting requirements do not impose a disproportionate burden on any size of project or organisations.
|
Data on:
-extent to which call planning schedules were respected
-time allowed for submission (and timing relative to holiday periods)
-time to award
-time to conclude grant agreement
-time to pay
Stakeholder perception of time allowed/taken for processes from publication of call to grant agreement, and of payment processes
Stakeholder perception of monitoring and report requirements
Stakeholder perceptions of Funding and Tenders portal
Stakeholder perceptions of eGrants system
|
Targeted survey of beneficiaries
Targeted survey of unsuccessful applicants
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
Deliberative workshops
|
|
13.To what extent did simplification measures proposed under the previous and current programmes generate an added value?
|
|
Answers provided under EQs 9-12 and case study 2
|
|
14.Is there any scope for further simplification in the management of the programme?
|
14.1 How can the programme management be further simplified, considering existing financial rules, to alleviate administrative burden on the Commission and on the applicants/beneficiaries?
|
Answers provided under EQs 9-12 and case study 2
|
|
Relevance
|
|
15.To what the extent is the programme still relevant in view of its objectives?
|
15.1 To what extent was the thematic focus of the calls published so far relevant to interested stakeholders?
15.2 To what extent has the programme integrated lessons learned from its predecessors?
|
The needs identified at the time the programme was adopted have not changed.
The needs identified at the time the programme was adopted were appropriate.
The programme has already demonstrated flexibility to adapt to changing needs.
The thematic focus of the calls so far was appropriate for the needs interested stakeholders aim to meet.
The findings on relevance of the mid-term evaluation of EfC and the interim and the first part of the ex post evaluation of the REC programme (as summarised in the introduction to this bid) were taken into account in the design of the programme.
|
Data on calls for proposals by theme and year (and specific objective)
Documentary evidence of a needs assessment (e.g. Impact Assessment, studies)
Findings on relevance from the evaluations of the predecessor programmes
Stakeholder perceptions that the programme is addressing needs that should be priorities for achieving its objectives
Stakeholder perceptions that the themes addressed in the calls so far are suitable for meeting the needs they seek to address
Stakeholders’ perceptions that the findings of the evaluations of the predecessor programme have been taken into account
|
Desk research
Targeted survey of beneficiaries
Public consultation
Interviews with beneficiaries
Focus group with NCPs
Focus group with Programme Committee Members
Deliberative workshops
Expert interviews
|
|
16.To what extent do the needs addressed by the programme continue to require action at EU level?
Answer provided under EQ15 and EQs 22-23
|
16.1 Are the needs addressed by the programme relevant to the EU priorities?
|
There is an ongoing need for action at EU level to protect and promote rights and values as enshrined in the EU Treaties and the EU Charter by supporting CSOs and other stakeholders and encouraging civic and democratic participation.
Targeting these needs is appropriate in the light of the EU’s broader objectives of nurturing and sustaining a rights-based, equal, open, pluralist, inclusive and democratic society.
Targeting these needs is also consistent with the digital and green transitions.
|
Data on needs identified under EQ15
Stakeholder perceptions EU action is needed to address the needs addressed by the programme
Stakeholder perceptions of a correlation between the needs addressed by the programme and achieving the objectives of EU priorities
|
Desk research
|
|
17.How relevant are the groups targeted by the programme?
|
|
The programme is prioritising target groups appropriate to the achievement of its objectives.
All target groups are equally relevant and no groups are being left out.
|
Data from Regulations, IA and calls on target groups by strand and objective
Stakeholder perceptions that the programme is addressing the most appropriate target group per strand
Stakeholder perceptions of the balance across target groups and possible gaps
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Focus group with Programme Committee Members
Expert interviews
|
|
18.Is the programme likely to continue being relevant for EU citizens in the near future? – answer provided under EQ15
|
18.1 Since its introduction, how well adapted is the programme to the pace of progress generated by digital technologies?
|
The need to reach out to EU citizens to promote EU values and support civil society in promoting citizen engagement with EU values, civic and democratic life is unlikely to change in the next two-to-three years.
The programme has been able to keep up with the pace of digitalisation in its programme and communication platforms and tools, and in encouraging digitally advanced actions.
|
Evidence of long-term results which will continue to be relevant to EU citizens in the future
Stakeholder perceptions that the programme will continue to be relevant for EU citizens in the near future.
Examples that digital technologies are well integrated in the programme.
|
Desk research
Interviews with beneficiaries
Targeted survey of beneficiaries
Focus group with Programme Committee Members
Expert interviews
|
|
Coherence
|
|
19.To what extent is the programme still coherent with other EU and national policies and funding programmes that have similar objectives?
|
|
The programme complements or has developed synergies with EU policies such as the Justice programme, the AMIF, the Internal Security Fund, the European Regional Development Fund, the Creative Europe programme and Erasmus+; the European Social Fund+; and Horizon Europe, in particular its Cluster 2 on Culture.
The programme complements or has developed synergies with the policies and funding programme of major international and national foundations promoting citizens’ rights, democracy and/or providing structural support to civil society (e.g. International Foundation for Better Governance, Stefan Batory Foundations), other international and national government-backed funds (e.g. Nordic Council of Ministers, the Visegrad Fund, Europe, EEA/Norway Grants Active Citizens Fund, Franco-German Citizens Fund).
|
Documentary evidence of a structured approach to avoiding overlaps and duplication with other policies and funding programmes
Stakeholder perception of a structured approach to or de facto avoidance of overlaps and duplication with other policies and funding programmes
Stakeholder perception that the CERV programme aligns well with how other funding programmes see needs developing and that it occupies an appropriate place in the landscape
|
Targeted survey of beneficiaries
Targeted survey of unsuccessful applicants
Targeted survey of NCPs
Focus group with Programme Committee Members
Public consultation
Case study 5
|
|
20.To what extent have the various components of the programme generated synergies and/or compensated possible trade-offs amongst them?
|
|
EACEA/DG JUST have policies and practices for maximising complementarities between the strands of the CERV programme.
Where there is a possibility of overlap between strands or calls, EACEA/DG JUST have mechanisms in place for avoiding these, including via trade-offs across them.
|
Documentary evidence of a structured approach to synergies and complementarities across strands and calls
Stakeholder perceptions of coherence between the components of the programme
EC stakeholder perceptions that any trade-offs are equitable and benefit the programme
|
Targeted survey of beneficiaries
Interviews with beneficiaries
Desk research
|
|
21.To what extent is the programme coherent with international commitments and objectives, including the Sustainable Development Goals?
|
|
programme calls take into account international agreements and standards relevant to the thematic topic, e.g. the UN Convention on the Rights of the Child.
The programme is achieving satisfactory results in EU budget performance measurement on the horizontal priorities on green budgeting (climate mainstreaming, biodiversity mainstreaming, clean air) and climate change.
The programme is achieving satisfactory performance on Sustainable Development Goals 4, 5, 8, 10, 16 and 17 as required by EU performance budgeting.
|
Stakeholder perceptions that calls take all appropriate international agreements and standards into account and do not include any that are not relevant
EU budget performance scoring on climate change and SDGs relative to other citizen-oriented programmes, e.g. ESF, Erasmus+, Creative Europe
Stakeholder perceptions of alignment between the programme and the Sustainable Development Goals
|
Desk research
Targeted survey of beneficiaries
Interviews with beneficiaries
|
|
EU Added value
|
|
22.Are the subsidiarity arguments put forward in the prior impact assessment still valid?
|
22.1In the absence of EU level action, to what extent would Member States have the ability or possibility to put in place appropriate measures?
22.2What is the additional value resulting from the programme compared to what could have been reasonably achieved (in terms of efficiency and effectiveness) by Member States acting alone?
|
Only some Member States have the same level of political will, financial capacity and administrative capacity to provide funding to achieve the same objectives as exists at EU level at both national and transnational level.
Only some Member States are achieving similar results from equivalent funding programmes but that added value is unevenly spread across the EU and cannot therefore compensate to action at EU level to level the playing field.
|
Examples of actions that could not have been funded / obtained the same results without this programme
Stakeholder perceptions of the ability of Member States to achieve comparable results across the EU at the same level of effectiveness and efficiency
Examples of additional value in relation to the programme compared to what could have been reasonably achieved (in terms of efficiency and effectiveness) by Member States acting alone
|
Targeted survey of beneficiaries
Interviews with beneficiaries
Deliberative workshops
Public consultation
|
|
23.What would be the most likely consequences of stopping the programme?
|
23.1 What would be the most likely consequences of not proposing a follow-up programme under the next MFF?
|
There are reasons to conclude that if the programme were either halted now or no follow-up were proposed under the next MFF, the following would occur in some or many Member States:
·the civic space would shrink significantly;
·respect for the rule of law, fundamental rights & democratic dialogue, transparency & good governance would decline significantly;
·equality, reduced discrimination & improved responses to racism & xenophobia, inc. hate speech & hate crime, both on- and offline, with particular attention having been given to addressing the vulnerabilities of specific groups & communities particularly affected by discrimination, would be achieved more slowly or go into reverse;
·civic and democratic participation would develop significantly more slowly or go into reverse;
·the understanding of EU, its origins, purpose, diversity, achievements, values and of the importance of mutual understanding and tolerance Increased awareness, attitudinal change would be significantly undermined;
·early detection, prevention of violence, and strengthened victim protection and support would progress more slowly or go into reverse.
The long-term impacts of actions under this programme and its predecessors would weaken without a supportive environment
|
Evidence from preceding EQs
Stakeholder perceptions, including perceptions of the extent to which there might be other funders who would step in to fill any gap left by CERV
|
Public consultation
Targeted survey of beneficiaries
Interviews with beneficiaries
Deliberative workshops
Expert interviews
|
Cross-cutting evaluation questions matrix
|
Evaluation question
|
Sub-question
|
Judgement criteria (There is evidence that …)
|
Indicators
|
|
Effectiveness
|
|
To what extent are there differences in the extent to which the policy objectives have been / are being achieved and why?
|
To what extent in the light of the conclusions from the evaluations of the REC, the EfC and CERV should the general and specific policy objectives of any future programme be modified?
|
The general objectives of the CERV programme can be carried forward to a future programme without any changes.
The specific objectives of the CERV strands can be carried forward to a future programme without any changes or redistribution across strands.
|
Findings and conclusions on the effectiveness of the REC, the EfC and CERV programmes based on the answers to the respective programmes’ evaluations’ questions.
|
|
Efficiency
|
|
To what extent has the new programme architecture produced efficiency gains?
|
To what extent did simplification measures proposed under the previous and current programmes reduce costs and increase benefits?
How efficiently has the CERV programme used human and financial resources compared with REC and EfC?
Has the Commission achieved economies of scale as a result of the new architecture?
|
The accompanying ex post evaluation of the EfC has demonstrated the added value of lump sums and unit costs.
The introduction of lump sums for actions under the equality, rights and gender equality strand, the Citizens’ engagement and participation strand, the Daphne strand and for actions funding programme Contact Points is producing efficiency gains for the European Commission and beneficiaries.
The streamlining of the number of indicators has not led to any loss of efficiency (or effectiveness).
The programmes had sufficient human and financial resources (inputs) to achieve their objectives.
A single architecture has simplified and streamlined management of the programme.
|
Findings and conclusions on the efficiency of the REC, the EfC and CERV programmes based on the answers to the respective programmes’ evaluations’ questions.
|
|
Relevance
|
|
To what extent did bringing all strands within the architecture of the CERV programme increase the relevance of the programme in view of its objectives?
|
To what extent has the CERV programme integrated lessons about needs from its predecessors?
To what extent is the CERV programme more relevant to the needs of the targeted groups than its predecessors?
To what extent are there currently unmet or likely emerging needs that should be taken into account in any future programme
|
The findings on relevance of the mid-term evaluation of EfC and the interim and the first part of the ex post evaluation of the REC were taken into account in the design of the programme.
The CERV programme is now more closely aligned to the needs of the stakeholders than its predecessors.
There are gaps in the needs coverage of the programme by theme or target group.
|
Findings and conclusions on the relevance of the REC, the EfC and CERV programmes based on the answers to the respective programmes’ evaluations’ questions.
|
|
Coherence
|
|
How well have the synergies between the programmes been exploited, increasing the coherence of EU action under CERV?
|
To what extent are the strands of the CERV programme more coherent now than when EfC and REC were two separate programmes and there was no Values strand?
To what extent did the novelties under CERV contribute to greater coherence
|
The programme has eliminated the overlaps in certain areas that merging the EfC and REC was designed to avoid.
The new CERV programme architecture has:
- improved performance,
- lowered costs,
- reduced burdens,
- led to simplification,
- facilitated automation/digitalisation.
The novelties (e.g. cascading grants and use of an external comms agency, new IT systems for application) are all contributing to greater internal coherence.
|
Findings and conclusions on the coherence of the REC, the EfC and CERV programmes based on the answers to the respective programmes’ evaluations’ questions.
|
|
Value Added
|
|
What is the added value resulting from the new programme architecture?
|
To what extent has bundling equality, rights and values funding in a single programme led to better results than was achieved by the predecessor programme merging the EfC and REC into one programme and adding a Values strand
To what extent did the novelties under CERV add value
|
Having a single programme has produced results (outcomes) that Member States acting alone could not have achieved even more effectively and efficiently than the previous programmes.
The novelties (e.g. cascading grants and use of an external comms agency, new IT systems for application, expansion of the use of lump sums, unit costs) are increasing the effectiveness, efficiency, relevance and coherence of the programme.
|
Findings and conclusions on the EU added value of the REC, the EfC and CERV programmes based on the answers to the respective programmes’ evaluations’ questions.
|
Annex IV. Overview of benefits and costs, simplification and burden reduction
1.REC PROGRAMME
1.1.Overview of costs and benefits
Overview of costs and benefits identified in the study – REC programme
|
|
Citizens/Consumers
|
Businesses
|
Administrations
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Direct compliance costs (administrative costs)
|
One-off (application)
|
|
|
It is estimated that it takes an average of 41 man-days to complete the application (based on data provided by project coordinators for nine projects), and cost an average of EUR 53 508 (based on estimates provided by five beneficiaries).
|
|
|
One-off (management and reporting)
|
|
|
No quantitative or qualitative data available on the cost to beneficiaries of the management of awarded grants and related reporting to the EC.
|
|
Direct benefits
|
One-off (budget committed)
|
|
|
EUR 16.4 million
|
Budget committed to private non-profit organisations
|
EUR 61.5 million
|
Budget committed to public bodies
|
EUR 229.5 million
|
Budget committed to beneficiary type ‘Other not for profit organisation’
|
|
|
|
|
|
|
|
|
|
EUR 57.4 million
|
Budget committed to Higher education institutions
|
|
|
|
|
|
|
|
|
|
EUR 45.1 million
|
Budget committed to research organisations
|
|
|
Recurrent
|
No quantitative data available on the recurrent benefits of the programme to citizens.
|
Direct benefits as reported by beneficiaries:
-increased visibility and awareness on the different rights covered by the programme;
-increased/improved support provision to wide range of target groups.
|
No quantification possible. Direct benefits as reported by beneficiaries (businesses, public institutions, CSOs and other types of beneficiaries):
-availability of independent funding;
-strengthening the capacity of civil society across the EU;
-increasing cross-border cooperation;
-increased visibility and awareness on the different rights covered by the programme;
-empowerment and support to wide range of target groups.
|
|
Indirect benefits
|
One-off
|
EUR 409.8 million
|
Total REC committed budget benefiting target groups
|
|
|
|
|
|
|
|
|
Recurrent
|
n/a; only data available on economic benefits of improved gender equality in the EU – EUR 1.95 – EUR 3.15 trillion by 2050
|
|
|
|
|
|
|
|
1.2.Simplification and burden reduction
Simplification and burden reduction already achieved – REC programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Digitalisation of the application process through the use of eGrants
|
Not applicable
|
|
No quantitative data available on burden reduction for applicants
|
Digitalisation has led to resource savings in paper and postal costs
|
No quantitative data available on burden reduction for applicants
|
Digitalisation has led to resource savings in paper and postal costs
|
No quantitative data available on burden reduction for applicants
|
Digitalisation has led to resource savings in paper and postal costs
|
Table 9: Potential simplification and burden reduction – REC programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Simplification of application requirements & better guidance on use of eGrant portal
|
|
One-off
|
|
|
Quantitative data not available. A significant proportion of beneficiaries and non-successful applicants still raised concerns about the heavy administrative burden related to the application process, in part due to the complexity of the application, terminology used, annual applications of the OGs, and difficulties in using the eGrant system. A few CSO beneficiaries noted that a two-step application process would allow for a preliminary evaluation, reducing the number of full applications that need detailed review and enabling organisations to focus on more promising opportunities (i.e. more likely to be successful). However, no cost effectiveness analysis was carried out on this option.
|
|
Simplifying financial reporting and providing clearer guidance on performance indicators
|
|
Recurrent
|
|
|
Quantitative data not available. Beneficiaries surveyed and interviewed noted that streamlined processes and clear instructions would reduce the administrative burden and enhance the accuracy and timeliness of reports.
|
2.EfC PROGRAMME
2.1.Overview of costs and benefits
Overview of costs and benefits identified in the study – EfC programme
|
|
Citizens/Consumers
|
Businesses
|
Administrations
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
|
One-off (application)
|
|
|
No quantitative data. Qualitative data suggests that the application costs can be estimated up to 1-2 FTE for 1 month for the coordinator organisation and ad hoc support by partner organisations as applicable.
|
|
|
One-off (management and reporting)
|
|
|
Estimates for the different groups based on mean and median project funding committed, assuming up to 10% of budget is spent on management and reporting (qualitative data):
·For public bodies – mean = 31 589, median = 25 000
·For orgs representing local authorities – mean = 38 738, median = 25 000
·For other types – mean = EUR 58 117, median = 25 0000
|
|
Direct benefits
|
One-off
|
|
|
|
|
Total budget committed to:
·Remembrance actions and Operating grants: EUR 35 600 585
·Civil society actions and Operating grants: EUR 132 878 049
|
|
|
Recurrent
|
|
|
No quantification possible. Direct benefits as reported by beneficiaries, public institutions, CSOs and other types of beneficiaries):
·ability to implement desired approaches
·increased networking with partners and outreach to new stakeholders
·enhanced visibility at national and EU level
·prestige and enhanced credibility in front of other donors
·testing of new approaches to address societal challenges
·greater capacity for advocacy actions
·sustainability of organisations
|
|
|
Recurrent
|
Raised awareness of remembrance, the common history and values of the Union and the Union’s aim for 1 724 932 citizens between 2014 – 2020
|
Direct participants in Remembrance strand
|
|
|
|
|
|
|
|
|
|
Encouraged the democratic and civic participation of 7 647 859 citizens between 2014 - 2020
|
Direct participants in Civil society strand
|
|
|
|
|
|
|
|
Indirect benefits
|
One-off
|
Total budget committed benefiting target groups:
·Remembrance actions and Operating grants: EUR 35 600 585
·Civil society actions and Operating grants: EUR 132 878 049
|
|
|
|
|
|
|
|
|
|
|
>60 million between 2014 - 2020
|
Indirect participants in Remembrance strand
|
|
|
|
|
|
|
|
|
|
>300 million between 2014 - 2020
|
Indirect participants in Civil society strand
|
|
|
|
|
|
|
2.2.Simplification and burden reduction
Simplification and burden reduction already achieved – EfC programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Use of lump sums and unit costs led to reduction of the administrative burden
|
n/a
|
|
n/a
|
Use of lump sums and unit costs led to reduction of the administrative burden
|
n/a
|
Use of lump sums and unit costs led to reduction of the administrative burden
|
n/a
|
Use of lump sums and unit costs led to reduction of the administrative burden
|
|
Creation of a single finance unit replacing the embedding of finance officers in each policy implementation unit at EACEA improved programme management
|
n/a
|
|
n/a
|
Creation of a single finance unit replacing the embedding of finance officers in each policy implementation unit at EACEA improved programme management
|
n/a
|
Creation of a single finance unit replacing the embedding of finance officers in each policy implementation unit at EACEA improved programme management
|
n/a
|
Creation of a single finance unit replacing the embedding of finance officers in each policy implementation unit at EACEA improved programme management
|
Table 12: Potential simplification and burden reduction – EfC programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Simplification of application process & use Funding and Tenders Portal
|
|
One-off
|
|
|
Quantitative data not available. Qualitative data suggest some dissatisfaction, while small, remained as regards the ease of using the application system and the ease of using the Funding and Tenders Portal.
|
|
Simplification of financial reporting and providing clearer guidance on performance indicators
|
|
Recurrent
|
|
|
While most beneficiaries had no particular difficulty collecting the data and matching it to the indicators, newcomers and less experienced organisations struggled to provide the estimates they had been expected to make on the participant data.
|
3.CERV PROGRAMME
3.1.Overview of costs and benefits
Overview of costs and benefits identified in the study – CERV programme
|
|
Citizens/Consumers
|
Businesses
|
Administrations
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Direct compliance costs (administrative costs)
|
One-off
|
|
|
No quantitative or qualitative data available on application costs.
|
1-2 FTE for 1 month
|
Application costs – estimate based on qualitative data; only available for CSOs
|
|
|
One-off
|
|
|
approx.
EUR 10 299
|
Reporting costs – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 102 987
|
approx.
EUR 5 047
|
Reporting costs – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 50 468
|
approx.
EUR 30 173
|
Reporting costs CSOs – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 301 727
|
|
|
|
|
|
|
|
|
|
approx.
EUR 19 477
|
Reporting costs Research organisations – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 194 766
|
|
|
|
|
|
|
|
|
|
approx.
EUR 20 735
|
Reporting costs Higher education institutions – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 207 344
|
|
|
|
|
|
|
|
|
|
approx.
EUR 39 663
|
Reporting costs International organisations – estimated based on survey feedback that up to 10% of project budget is spent on reporting, with average project budget ca. EUR 396 627
|
|
Direct benefits
|
One-off
|
|
|
EUR 6 102 144.91
|
Budget committed to private non-profit organisations
|
EUR 28 762 064.00
|
Budget committed to public bodies
|
EUR 258 709 334.97
|
Budget committed to CSOs
|
|
|
|
|
|
|
|
|
|
EUR 14 096 265.00
|
Budget committed to Higher education institutions
|
|
|
|
|
|
|
|
|
|
EUR 26 456 015.00
|
Budget committed to research organisations
|
|
|
|
|
|
|
|
|
|
EUR 1 734 108.73
|
Budget committed to international organisations
|
|
|
Recurrent
|
|
|
No quantification possible. Direct benefits as reported by beneficiaries (businesses, public institutions, CSOs and other types of beneficiaries):
·ability to implement desired approaches
·increased networking with partners and outreach to new stakeholders
·enhanced visibility at national and EU level
·prestige and enhanced credibility in front of other donors
·testing of new approaches to address societal challenges
·greater capacity for advocacy actions
·sustainability of organisations
|
|
|
|
ca. EUR 309 million
|
Improved awareness of policy areas covered by CERV among an estimated 39 160 000 EU citizens: 89% of surveyed participants reported increased awareness about policy areas addressed through project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
|
|
|
Increased knowledge about the policy areas covered by CERV among an estimated 38 280 000 EU citizens: 87% of surveyed participants reported increased knowledge about the policy areas addressed through project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
|
|
|
Increased skills related to policy areas covered by CERV among an estimated 31 120 000 EU citizens: 73% of surveyed participants reported increased skills related to policy areas addressed through project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
|
|
|
Increased cooperation among an estimated 35 640 000 EU citizens: 81% of surveyed participants reported improved cooperation as a result of project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
|
|
|
Change in behaviour among an estimated 36 960 000 EU citizens: 84% of surveyed participants reported a change in behaviour as a result of participation in project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
|
|
|
Change in practice among an estimated 40 480 000 EU citizens: 92% of surveyed participants reported a change in participants’ practice as a result of participation in project activities; an estimated 44 million people have been reached or engaged through the programme
|
|
|
|
|
|
|
|
Recurrent
|
n/a; only data available on economic benefits of improved gender equality in the EU – EUR 1.95 – EUR 3.15 trillion by 2050
|
Civil society strengthened – governments held to account, promotion of voices of local and marginalised communities, rule of law strengthened, respect of fundamental rights improved, service provision for vulnerable groups improved, transparency and data improved.
|
|
|
|
|
|
|
3.2.Simplification and burden reduction
Simplification and burden reduction already achieved – CERV programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other programme beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Multiannual work programmes
|
|
Recurrent
|
|
|
n/a
|
Predictability of the programme has improved, facilitating planning
|
n/a
|
Predictability of the programme has improved, facilitating planning
|
n/a
|
Predictability of the programme has improved, facilitating planning for CSOs, research organisations, higher education institutions, and international organisations
|
|
Streamlining of programme performance indicators
|
|
One-off
|
|
|
n/a
|
There are less indicators to reflect in application and reporting forms, which might constitute time savings, albeit no evidence to substantiate this assumption
|
n/a
|
There are less indicators to reflect in application and reporting forms, which might constitute time savings, albeit no evidence to substantiate this assumption
|
n/a
|
There are less indicators to reflect in application and reporting forms, which might constitute time savings, albeit no evidence to substantiate this assumption
|
|
Digitalisation of the application process through the use of eGrants
|
|
One-off
|
|
|
n/a
|
Digitalisation has led to resource savings in paper and postal costs
|
n/a
|
Digitalisation has led to resource savings in paper and postal costs
|
n/a
|
Digitalisation has led to resource savings in paper and postal costs
|
|
Reduction in threshold of applications to be selected from 80 to 70 points
|
|
Recurrent
|
|
|
n/a
|
There may be less of a deadweight cost of submitting unsuccessful applications under Strands 1 and 3 albeit lack of evidence to substantiate this assumption; oversubscription, particularly under Strands 2 and 4 continues to create a large deadweight cost to the sector.
|
n/a
|
There may be less of a deadweight cost of submitting unsuccessful applications under Strands 1 and 3, albeit lack of evidence to substantiate this assumption; oversubscription, particularly under Strands 2 and 4 continues to create a large deadweight cost to the sector.
|
n/a
|
There may be less of a deadweight cost of submitting unsuccessful applications under Strands 1 and 3, albeit lack of evidence to substantiate this assumption; oversubscription, particularly under Strands 2 and 4 continues to create a large deadweight cost to the sector.
|
Table 15: Potential simplification and burden reduction – CERV programme
|
|
Citizens/Consumers
|
Businesses
|
Administration
|
Other beneficiaries
|
|
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
Quantitative
|
Comment
|
|
Limiting repetitive elements in applications and reporting forms
|
|
One-off
|
|
|
ca. 1 FTE for 2 days
|
This assumption is based on qualitative feedback that removing repetitive elements could reduce complexity and improve the application and reporting experience
|
ca. 1 FTE for 2 days
|
This assumption is based on qualitative feedback that removing repetitive elements could reduce complexity and improve the application and reporting experience
|
ca. 1 FTE for 2 days
|
This assumption is based on qualitative feedback that removing repetitive elements could reduce complexity and improve the application and reporting experience
|
|
Providing flexibility in reporting for beneficiaries of operating grants to align with their financial year
|
|
Recurrent
|
|
|
n/a
|
n/a
|
n/a
|
n/a
|
ca. 0.5 FTE for 1 day
|
This concerns operating grants beneficiaries - assumption is based on the estimated time that it takes to request extensions and correspondence with the Commission on reporting delays
|
Annex V. Stakeholders consultation - Synopsis report
Contents
1.INTRODUCTION
2.OBJECTIVES OF THE CONSULTATION
3.CONSULTATION ACTIVITIES AND METHODOLOGY
4.RESULTS
4.1. Call for evidence
4.2. Targeted survey of beneficiaries:
4.3. Targeted survey of unsuccessful applicants:
4.4. Targeted survey of third-party beneficiaries:
4.5. Interviews with beneficiaries:
4.5.1.
Interviews with EfC beneficiaries:
4.5.2.
Interviews with REC beneficiaries:
4.5.3.
Interviews with CERV beneficiaries:
4.6. Focus groups with Programme Committee Members:
4.7. Consultations with EfC and CERV NCPs:
4.8. Deliberative workshops with stakeholders and citizens
4.9. Public consultation
4.10. Expert interviews
1.INTRODUCTION
Two successive generations of EU programmes have pursued the goal of protecting and promoting the rights and values enshrined in the EU Treaties and the Charter of Fundamental Rights: the 2014-2020 ‘Rights, Equality and Citizenship’ (REC) and ‘Europe for Citizens’ (EfC) programmes, and the 2021-2027 ‘Citizens, Equality, Rights and Values’ (CERV) programme. The REC programme aimed to contribute to the further development of an area where equality and the rights of persons, as enshrined in the EU Treaty, the Charter and international human rights conventions, are promoted and protected. In parallel, the EfC programme supported initiatives to strengthen remembrance of recent European history and to enhance civic participation at EU level. Since 2021, the CERV programme has sought to support and develop open, democratic, equal, inclusive and rights-based societies based on the rule of law. In this context, and as required by legislation, the European Commission conducted an evaluation in three parts:
(1) an ex post evaluation of the 2014-2020 of the REC programme, assessing its longer-term impact and sustainability effects, based on the results of the first part of the ex post evaluation completed in 2022;
(2) an ex post evaluation of the 2014-2020 EfC programme, assessing the results achieved, and its long-term impact and sustainability effects;
(3) an interim evaluation of the 2021-2027 CERV programme, assessing its preliminary achievements.
The combined ex post and interim evaluation of the three programmes aimed to identify how the current programme (CERV) took account of lessons learned under its predecessors, and how results can be used for further improvement and to feed into the next funding cycle.
2.OBJECTIVES OF THE CONSULTATION
The purpose of the consultation was to strengthen the evidence base for the evaluation by gathering the views of all relevant stakeholders on the REC, EfC and CERV programmes’ effectiveness, efficiency, relevance, coherence, and EU added value.
3.CONSULTATION ACTIVITIES AND METHODOLOGY
The stakeholder consultation was conducted using a mixed-methods approach. The Commission conducted a call for evidence feedback process open to the public and stakeholders. The stakeholder consultation included six surveys – a targeted survey of beneficiaries, a targeted survey of unsuccessful applicants, a survey of NCPs, a beneficiary survey on costs, a survey of ‘third-party beneficiaries’, meaning recipients of CERV funding via intermediaries (the re-granting mechanism), as well as a public consultation open to the public and stakeholders. The qualitative data collection involved interviews with beneficiaries, focus groups with NCPs for EfC and CERV, focus groups with Programme Committee Members of all three programmes, and deliberative workshops with EU networks, national stakeholders, and citizens, as well as six expert interviews. Over 1 000 stakeholders were consulted in total. In addition, the CERV interim evaluation included five case studies.
As outlined in the consultation strategy published on the ‘Have your say’ website of the European Commission, stakeholders consulted during the evaluation included:
·target groups of funded activities: the public, participants in funded activities.
·applicants and beneficiaries of the funding programmes: EU networks, platforms and professional associations (including the European Network of Equality Bodies); CSOs; foundations; public authorities; private and public universities and other research organisations active in the fields covered by the programmes; private companies; international organisations.
·other influential actors: EU Member States that are also members of the relevant Programme Committees; expert groups assisting the European Commission’s Directorate-General for Justice and Consumers; agencies (e.g. the European Education and Culture Executive Agency, the EU Agency for Fundamental Rights); European Commission services and other relevant mechanisms responsible for the complementary instruments; CERV and EfC NCPs.
Details of the consultation activities by funding programme are outlined below:
|
REC programme
|
Details
|
|
Public consultation
|
27 respondents familiar with REC, and 12 respondents had received REC funding; 6 general questions and 20 expert questions addressing all evaluation criteria.
|
|
Survey of beneficiaries
|
133 relevant responses from beneficiaries of REC to questions covering all evaluation criteria.
|
|
Survey of unsuccessful applicants
|
79 responses from unsuccessful applicants of REC to questions covering relevance, efficiency, coherence and added value.
|
|
Interviews with beneficiaries
|
50 interviews covering 20 Member States (except Croatia, Czechia, Denmark, Estonia, Luxembourg, Portugal, Slovakia), with balanced representation by type of grant and specific objective. 15 recurring beneficiaries in CERV.
|
|
Focus group with Programme Committee Members
|
2 focus group participants from Finland and Slovenia.
|
|
EfC programme
|
Details
|
|
Public consultation
|
24 respondents familiar with EfC, and three respondents had received EfC funding; 6 general questions, 20 expert questions addressing all evaluation criteria.
|
|
Survey of beneficiaries
|
235 relevant responses from beneficiaries of EfC to questions covering all evaluation criteria.
|
|
Survey of unsuccessful applicants
|
67 relevant responses from unsuccessful applicants of EfC to questions covering relevance, efficiency, coherence and added value.
|
|
Survey of NCPs
|
14 survey responses.
|
|
Interviews with beneficiaries
|
80 interviews across 24 EU Member States (except Cyprus, Estonia, Luxembourg) and Serbia, with balanced representation by type of grant and strand. 21 recurring beneficiaries in CERV.
|
|
Focus group with NCPs
|
9 focus group participants from five EU Member States (Estonia, France, Latvia, Romania and Sweden).
|
|
Focus group with Programme Committee Members
|
4 focus group participants from Croatia, Greece, Finland, Malta.
|
|
CERV programme
|
Details
|
|
Public consultation
|
39 respondents familiar with CERV, and 19 respondents had received CERV funding; 6 general questions, 20 expert questions addressing all evaluation criteria.
|
|
Survey of beneficiaries
|
272 relevant responses from beneficiaries of CERV to questions covering all evaluation criteria.
|
|
Survey of unsuccessful applicants
|
285 relevant responses from unsuccessful applicants of CERV to questions covering relevance, efficiency, coherence and added value.
|
|
Survey of NCPs
|
14 survey responses.
|
|
Survey of beneficiaries on costs
|
97 survey responses from 19 EU Member States (Austria, Belgium, Bulgaria, Croatia, Cyprus, Estonia, France, Germany, Greece, Italy, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovak Republic, Slovenia and Spain).
|
|
Survey of third-party beneficiaries
|
212 responses from 10 EU Member States (Bulgaria, Croatia, Denmark, Cyprus, Greece, Hungary, Lithuania, Poland, Slovenia and Spain).
|
|
Interviews with beneficiaries
|
97 interviews across 26 EU Member States (except Latvia), balanced representation by type of grant and strand, including recipients of contribution agreements (n=3).
|
|
Focus group with NCPs
|
15 focus group participants from 8 EU Member States (Estonia, France, Italy, Latvia, Netherlands, Portugal, Romania and Sweden).
|
|
Focus group with Programme Committee Members
|
10 focus group participants from Austria, Croatia, Estonia, Finland, France, Germany, Greece, Romania, Slovenia and Spain.
|
|
Deliberative workshops
|
14 EU networks working on policy areas under Strand 2 – Equality, rights and gender equality.
65 national stakeholders working in thematic areas under Strand 3 – Citizens’ engagement and participation from France, Greece, Hungary, Italy, Latvia, Poland.
164 citizens consulted on Strand 3 – Citizens’ engagement and participation thematic areas from Austria, France, Greece, Hungary, Ireland, Italy, Latvia, Poland, Romania and Sweden.
|
|
Expert interviews
|
6 expert interviews covering all four strands, which included independent experts working in the policy areas covered by the strands, as well as members of the European Cooperation Network and the EU Network for children’s rights.
|
|
Interviews with evaluators of CERV applications
|
4 interviews as part of case study on gender mainstreaming.
|
4.RESULTS
4.1.Call for evidence
There were 21 contributions to the Commission’s call for evidence, which ran from 22 August-19 September 2023. Feedback came from CSOs, EU networks and CERV NCPs. Eight contributions that did not comply with the European Commission’s rules for publishing feedback and were removed.
The feedback confirmed the essential contribution of the programme to supporting democracy and citizens across Europe and addressing the shrinking civic space and challenges to rule of law across Europe. It also confirmed the new programme architecture being viewed as ‘solid and meaningful’ with the four CERV strands. The evidence collected fed into answers on relevance and EU added value of the CERV programme.
Main points for consideration included the reinstatement of the Civil Dialogue Group, allowing for the co-financing of events in non-EU countries, increasing the lump sum to account for rises in cost-of-living expenses, as well as simplifying application processes, and increasing budgets for certain calls to allow for the achievements of expected impacts. Further simplification of the application process was also advocated.
4.2.Targeted survey of beneficiaries
Overall, 421 beneficiaries responded to the targeted survey. They came from all EU-27 Member States, as well as Albania, Bosnia and Herzegovina, North Macedonia, Serbia, and the United Kingdom. As
the f
igure
below
shows, CSOs and public authorities together accounted for almost two thirds of the responses (64%).
Figure A: What type of organisation do you represent? (%) (n=412)
Just over a quarter of respondents had been beneficiaries of either the EfC or the CERV programme (26%, 110 and 108 of 421, respectively). 17% (70 of 421) had been beneficiaries of both EfC and CERV; 11% (48 of 421) had been beneficiaries of both, REC and CERV; 11% (46 of 421) had been beneficiaries of all three programmes, and 9% (39 of 421) just of the REC programme.
A significant number of beneficiaries in all three programmes were recurrent beneficiaries of action grants.
Feedback collected through the targeted survey of beneficiaries fed into all evaluation criteria of all three evaluations, and – by way of summary – was as follows:
Funding received under the programmes generally aligned with the core needs of organisations, but 69% of action grant beneficiaries indicated this to be the case ‘to a large extent’ compared with 42% of operating grant beneficiaries. Most respondents (73%) reported that if they had received funding more than once, this was for a project or projects which were a continuation of an earlier project ‘to a large extent’ or ‘to some extent’. Most respondents (57%) also indicated that the project or projects for which they had received funding continued ‘to a large extent’ or ‘to some extent’ without Commission funding after it had ceased, although some noted in open comments that they had continued with less capacity and range than with EU funding.
Most respondents reported that their projects had achieved or were achieving their projected outcomes and results ‘fully’ or ‘to a large extent’ across all three programmes. In terms of developing synergies with projects funded by other EU sources, most (58%) reported that they had done this, particularly with projects funded under Erasmus+, Horizon Europe, and the European Social Fund, as well as AMIF and the European Parliament.
Most respondents (55%) indicated that they had integrated good practices from other REC/EfC/CERV-funded projects in another country: one fifth (22%) indicated that they had not done so. Innovations under CERV were rated highly by most respondents (over 70%), e.g. the publication of call schedules, fewer indicators, and lump sums. Less satisfaction was reported about unit costs. On the re-granting mechanism, 48% reported that they ‘did not know’ how to answer this question. It is likely that this is due to this mechanism only concerning a small proportion of CERV beneficiaries.
The vast majority of respondents (over 80%) reported that the three programmes had addressed appropriate needs and were likely to have a lasting impact. Most respondents (over 70%) also reported that they had contributed to the sustainability of organisations and/or projects and were well aligned with other EU funding programmes. Most respondents reported that they thought that the results of the programmes could only have been achieved by EU action ‘fully’ or ‘to a large extent’. For REC, this was the case for 79% of respondents, for EfC – 73%, and for CERV – 74%. EU funding was considered an ‘essential’ or ‘major contributor’ to a functioning civil society, relative to other sources of funding, by 68% of respondents.
|
Survey of beneficiaries on costs:
Results from this short survey are to be treated with caution, as there was no way to confirm the methodology that respondents applied when providing their estimates of average costs of activities in 2021, 2022 and 2023. However, it emerged that the average among spent on one awareness-raising, information and dissemination activity was EUR 8 842, for one mutual learning and exchange of good practice activity – EUR 10 597, for one training activity – EUR 8 842, and of one person trained per day – EUR 388.
|
4.3.Targeted survey of unsuccessful applicants
Overall, 343 unsuccessful applicants responded to the survey from all EU-27 Member States, as well as Israel, Kenya, Montenegro, Mozambique, Norway, Serbia, and Tanzania. An analysis of open comments on the reasons for failed applications confirmed that applicants from non-EU countries (other than Serbia) were aware – at least ex post – that their applications had not been eligible. Almost half the respondents were CSOs / NGOs working at national, local, or regional levels (49%, 162 of 332), followed by CSOs / NGOs working at international, transnational, or pan-European levels (16%, 53 of 332). The programme for which applicants had most frequently applied unsuccessfully was CERV (indicated 65% of the time), followed by REC (indicated 19% of the time) and EfC (indicated 15% of the time).
Feedback collected from the targeted survey of unsuccessful applicants mainly fed into the evaluation criteria efficiency, coherence and EU added value in the evaluations of all three programmes, and – by way of summary – was as follows. There were mixed responses on the usefulness of the feedback received on applications. Around one third of respondents indicated that it had been ‘very useful’ for the CERV and REC programmes, while 19% indicated the same for EfC. It was considered ‘partially useful’ by around one third of respondents under all three programmes. The feedback was considered ‘not very useful’ or ‘not useful at all’ by a third of respondents replying on the EfC, a quarter replying on CERV and 16% replying on REC. Unsuccessful applicants reported mixed effects of not having received REC/EfC/CERV funding on projects, depending on the programme. Under REC, respondents noted that 40% of the projects were cancelled, while 27% of respondents noted they reapplied and 8% stated they found other funding. Under CERV and the EfC, slightly lower proportions cancelled their projects (38% and 33%, respectively) and most projects either reapplied (27% and 25%, respectively) or found other funding (8% and 13%, respectively). Respondents who indicated that their project had received support from other funds most frequently mentioned that these came from foundations or CSOs (32% in the case of REC and CERV, 23% in the case of EfC). EU funds were also frequently indicated by a third of EfC beneficiaries, as were national funds by CERV beneficiaries (23%). Local and regional funds were mentioned by about a fifth of respondents across all three programmes. Most unsuccessful applicants (74%) reported that they would consider applying for funding under CERV in the future, and one fifth (20%) reported that they were ‘not sure’.
4.4.Targeted survey of third-party beneficiaries
Overall, 212 respondents participated in the survey of third-party beneficiaries (i.e. beneficiaries of grants re-granted through intermediaries under the CERV programme). They came from 10 EU Member States, with most from Poland (25%, 52 of 212), Bulgaria (21%, 45 of 212), and Greece (17%, 36 of 212). 69% of respondents (147 of 212) reported that they would describe their organisation as a ‘grassroot organisation’; 86% of respondents indicated that they worked with ‘vulnerable groups’, with most covering at least 2-4 such groups including older people, LGBTIQ+ people, migrants and refugees, people living in poverty, people with disabilities, Roma communities, women, young people and children.
Feedback collected from the targeted survey of third-party beneficiaries mainly fed into an accompanying case study on support provided to CSOs, and the evaluation criteria effectiveness, EU added value and relevance. By way of summary, this feedback was as follows:
·97% of respondents considered the funding they had received through the re-granting mechanism ‘essential’ and ‘important’ relative to all other funding sources;
·82% indicated that this funding had allowed them to achieve results that they would not have been able to achieve otherwise (16% responded ‘don’t know’ likely accounting for projects only having recently started);
·96% strongly agreed or agreed that the funding had enhanced their capacity to deliver projects, programmes and initiatives;
·90% strongly agreed or agreed that the funding had strengthened their sustainability as an organisation;
·91% indicated that the funding had enhanced their capacity for outreach, engagement and promotion of participatory processes;
·most respondents (74%) indicated that they rated the usefulness of the capacity-building activities that they had participated in (organised by intermediaries) as very useful and useful.
4.5.Interviews with beneficiaries
4.5.1.Interviews with EfC beneficiaries
80 interviews were conducted with former EfC beneficiaries across 24 EU Member States (except Cyprus, Estonia, Luxembourg), and Serbia, with a balanced representation by type of grant received and strand. There were 21 recurring beneficiaries in CERV. Evidence from the interviews fed into all evaluation criteria of the EfC ex post evaluation, and – by way of summary – was as follows:
·Visibility: The EfC had been well known to larger organisations with prior experience of EU funding. Some beneficiaries perceived the promotion of the EfC to have been limited beneficiaries; this was found potentially to have caused a barrier to first-time applicants or smaller organisations.
·Relevance: There was a consensus among beneficiaries that the programme’s priorities had aligned well with the greatest needs in the Member States relating to civic engagement and remembrance.
·Application process: Beneficiaries had welcomed the simplifications which had taken place over the successive iterations of the programme, although there was some limited negative feedback that certain aspects of the application process had become overcomplicated.
·Selection process: The selection process was reported as fair and transparent overall, and it was noted that appropriate feedback had been given on the outcomes in most cases, though many would have liked to see more detail provided on the areas for improvement.
·Approach to financing: Beneficiaries had welcomed the introduction of lump sums and unit costs, since it had facilitated the cost reporting and simplified the project management processes. Several beneficiaries advised that a reduced co-financing rate for smaller CSOs would have enabled better access to the programme.
·Results: Results reported with potential for long-term impacts, included the establishment of lasting cooperation of stakeholders, citizens and networks of CSOs across the EU, increased fundraising and operational capacity of participating organisations, and critical funding for organisations which received an operating grant and the creation of sustainable outputs and project results (such as media, training materials, development of ‘train the trainers’ approaches) which can be re-used and disseminated.
·Flexibility: The programme was considered to have been flexible enough to respond to unexpected global challenges, such as COVID-19 and digitalisation.
·Added value: The funding received from the EfC programme had been an important support to their operations, but the majority did not see it as having been critical. However, the feedback highlighted that the funding allowed beneficiaries to achieve more than they would have been able to otherwise.
4.5.2.Interviews with REC beneficiaries
50 interviews were conducted with former REC beneficiaries covering 20 Member States (except Croatia, Czechia, Denmark, Estonia, Luxembourg, Portugal, Slovakia), with a balanced representation by type of grant received and specific objective. There were 15 recurring beneficiaries in CERV. Evidence from the interviews fed into all evaluation criteria of the ex post evaluation of REC.
Interviews with former beneficiaries of the REC programme confirmed the findings of the first part ex post evaluation, adding further nuance and examples of long-term effects of beneficiaries’ projects:
·Awareness: As regards general awareness of the programme, beneficiaries had mixed opinions on the extent to which the programme had been well-advertised in their sectors at the time but reported notable improvements around communication of DG JUST funding opportunities, particularly as regards the CERV programme. This was also attributed to the newly established NCPs.
·Flexibility: Former beneficiaries were generally satisfied with the programme’s flexibility, particularly in the context of the COVID-19 pandemic, which required significant adjustments in terms of project activities and timelines.
·Administrative burden: This was considered to have been an important cost component of the programme, with interviewees generally reporting that the application and reporting processes had been very burdensome and time-consuming. However, former beneficiaries still emphasised that the benefits of the programme had outweighed the cost.
·Results & impacts: Beneficiaries generally noted that they had met the objectives of their projects, or even overachieved. The main challenge identified for implementation was COVID19 and the resulting travel restrictions. The main long-term project impacts identified included the creation of lasting relations with relevant stakeholders, increased awareness of the project outcomes among policy makers, increased awareness about the project topics among target groups, increased internal capacity (operating grants), as well as the continued existence and access to project outputs.
·Overlaps: Beneficiaries did not note any overlap in REC priorities and topics compared with other funds. The exception was the Daphne strand where some thematic similarities were noted compared with the Justice programme.
·Added value of REC: Lastly, there were strong opinions about the EU being a very important funding source, especially for national or European civil society organisation, and that the implementation of beneficiaries’ projects would not have been possible without REC funding.
4.5.3.Interviews with CERV beneficiaries
97 interviews were conducted with CERV beneficiaries across 26 EU Member States (except Latvia), with a balanced representation by type of grant received and strand, including three recipients of contribution agreements. Evidence from the interviews fed into all evaluation criteria of the interim evaluation of CERV. The feedback overwhelmingly confirmed that the CERV programme was unique, much needed and welcomed by beneficiaries, and compared with the two predecessor programmes, progress was noted in several areas:
·Visibility: The CERV programme was generally considered well-known and well-communicated among larger organisations; positive views on communication were mainly linked to the establishment of NCPs. However, beneficiaries noted that the CERV programme had not managed to build a community around the programme (yet) that would regularly meet, exchange ideas, and shape programme developments.
·Predictability: The two-year work programmes were found to have improved predictability in anticipating calls, but beneficiaries noted that not all policy areas were covered by calls in all years, leading to long gaps for organisations to be able to reapply if unsuccessful.
·Gender mainstreaming: While there was consensus that the increased gender mainstreaming requirements were a positive development, some beneficiaries found them challenging and would have appreciated more guidance on this topic.
·Indicators: While recurrent beneficiaries from REC appreciated the reduced number of indicators, those from the EfC found that the reporting process had become more complex.
·Lump sums and unit costs: While lump sums were generally welcomed for their flexibility and simplified financial reporting, some beneficiaries felt that they incentivised ‘inventing events that do not need to exist’. By contrast, unit costs garnered predominantly negative feedback, as they were found to complicate cooperation with project partners and internal accounting processes and were deemed inappropriately low compared with market prices.
·Re-granting mechanism: Both intermediaries (action grants) and those operating grant beneficiaries who used re-granting noted that they had very much welcomed this development. It was deemed to provide access to EU funds for small organisations that might have otherwise abstained from applying directly to the Commission for fear of facing high competition and administrative burden. Suggestions for improvement included more peer learning.
4.6.Focus groups with Programme Committee Members
Three online focus groups were conducted with 13 Programme Committee Members of all three programmes. Findings from these particularly fed into the evaluation criteria effectiveness, EU added value and relevance of the ex post evaluation of the EfC and interim evaluation of CERV. Given the limited feedback from REC Programme Committee Members, to ensure confidentiality of responses, these were excluded from the analysis. By way of summary, the evidence was as follows:
·Former EfC Programme Committee Members and CERV Programme Committee Members generally felt that there had been sufficient communication with the Commission, however, the latter noted that earlier receipt of documents would (have) enabled greater consultation with national stakeholders and thus more detailed feedback on drafts of work programmes.
·The EfC was considered to have been a popular programme, particularly the Remembrance and Town Twinning and Networks of Towns strands. Scope to improve communication about CERV was noted, particularly its visibility and visual identity, which could be aided by a logo. NCPs were considered a key strength of the CERV programme for improving awareness at national and local levels.
·While under the EfC, the Town Twinning and Networks of Towns strand was deemed to have been undervalued given its popularity and accessibility to small municipalities. Under CERV, Strand 2 – Equality, rights and gender equality was felt to require a significant increase in budget to account for the number and seriousness of policy areas covered.
·Former EfC Programme Committee Members noted that programme implementation had been successful overall but emphasised that lump sums and co-financing requirements might have been ill-suited or overly burdensome for small organisations.
·As regards CERV, Programme Committee Members noted further scope to improve the programme architecture in terms of coherence and streamlining of the structure, to make it easier for new applicants to navigate the programme. Other CERV developments, such as the re-granting mechanism, were considered positive in expanding the programme’s reach to grassroot organisations, while the two-year work programmes were felt to have improved predictability of the programme overall.
·Former EfC Programme Committee Members noted similarities between EfC, Creative Europe and Erasmus+, but emphasised that the EfC had supported very specific types of activities and that there were no overlaps between the programmes. Similarly, CERV Programme Committee Members called the CERV programme ‘unique’ in its priorities and types of activities funded. However, they felt that the similarity of topics covered by AMIF, ESF+ and Erasmus+ warranted further exploration of synergies between these programmes.
4.7.Consultations with EfC and CERV NCPs
A survey of NCPs was conducted, which garnered 14 responses. Respondents were not asked about the country in which they were based to ensure anonymity. Additionally, two online focus groups were conducted that were joined by NCPs based in 8 EU Member States, namely Estonia, France, Italy, Latvia, Netherlands, Portugal, Romania and Sweden. Evidence from the survey and focus groups with NCPs fed into answers to all evaluation criteria for both the ex post evaluation of EfC and the interim evaluation of CERV. By way of summary, the evidence presents as follows:
·Compared with the EfC, which participants felt had managed to achieve wide recognition among NGOs and municipalities, the CERV programme was felt to still require extensive communication efforts to achieve the same popularity.
·Communication activities have centred on digital and social media communication, as well as direct outreach. This last was considered crucial to reach new organisations with less experience in EU funds, particularly in remote regions.
·Questions from potential applicants mainly centred on the financial aspects of application processes and came from organisations with less experience with EU funding and writing proposals.
·There was consensus that the Commission has been very supportive and had an adequate infrastructure in place to address any questions quickly and usefully, although more access to data would be welcomed to allow NCPs to have better insights into the functioning of the programme in their country to aid targeted outreach and support to unsuccessful applicants.
·Specific barriers to increasing the number and quality of applications included guidance documents being predominantly in English, limited knowledge and capacity of small organisations and municipalities to develop quality proposals, the reputation of the CERV application process being burdensome and complex, as well as technological barriers in using the EU Funding & Tenders Portal.
·Innovations under CERV were felt to have been welcomed by all interested parties for improving accessibility to small organisations (re-granting mechanism), reduced monitoring burden, greater focus on results reporting, predictability of the programme, and improved gender equality.
4.8.Deliberative workshops with stakeholders and citizens
44 deliberative workshops were conducted with stakeholders and citizens which fed into the EU added value, relevance and effectiveness assessment of the CERV programme. The stakeholders consulted included EU networks (specifically working within the policy areas covered by Strand 2 of CERV – Equality, rights and gender equality) and national stakeholders working in the areas covered by Strand 3 of CERV – Citizens’ engagement and participation. Citizens were also consulted on the topics covered by Strand 3. Overall, 14 EU networks, 65 national stakeholders and 164 citizens participated in the deliberative workshops – stakeholders and citizens came from Austria, Greece, Hungary, Italy, Ireland, Latvia, France, Poland, Romania, and Sweden.
There was consensus among participants that the CERV programme and EU funding more broadly bring significant added value, especially considering the lack of national funding opportunities of similar value. While ‘more is always better’, participants generally considered CERV is already an ideal EU programme in terms of the type and amount of the funding opportunities. However, the programme was considered less ideal with regard to the application process due to the high burden it placed on applicants, in particular for smaller and grassroot organisations. There was also consensus that there was a great need for the CERV programme to continue and grow in funding available to address pressing societal needs.
Citizens also perceived that the CERV programme had a clear added value as it addressed gaps at the national level – most found that their national governments would not fund the types of projects funded by CERV, or at least not to the same extent and certainly not on a transnational scale. They also found that the CERV programme provided for more ‘objective’ projects in the sense of not being politically motivated or driven by national interests. They considered the thematic areas covered under Strand 3 to be important and complementary but gave mixed responses as to which they considered the most important. However, overall, civic engagement was rated as most important followed by town twinning projects as these were felt to engage citizens more directly and to have a more important impact in encouraging civic engagement, promoting democratic values, and strengthening the social fabric of their communities.
4.9.Public consultation
The public consultation was launched on 4 April 2024 on the ‘Have your say’ website of the European Commission and was open for 12 weeks (closing on 27 June 2024). The purpose of the consultation was to strengthen the evidence base for the evaluation of the REC, EfC and CERV programmes by gathering the views of relevant stakeholders on the programmes’ effectiveness, efficiency, relevance, coherence, and EU added value.
Overall, 53 contributions to the survey were received from stakeholders in 16 EU Member States. The largest proportion was from Belgium (26%, 14 of 53), followed by Italy (13%, 7 of 53), and Germany (11%, 6 of 53). More than half the respondents submitted their contributions as NGOs (55%, 29 of 53), followed by 21% (11 of 53) as EU citizens and 13% (7 of 53) as public authorities.
There were 40 contributions to the ‘expert part’ of the consultation, where respondents were asked to provide more detailed answers about the programmes – 34 respondents who contributed as an organisation, and six respondents who contributed as EU citizens. Most respondents were familiar with the programmes: 98% (39 of 40) with CERV, 68% (27 of 40) with REC, and 60% (24 of 40) with the EfC. Seventeen respondents, consisting of 15 NGOs, one business association, and one EU citizen, submitted a total of nine unique position papers in support of their contribution to the public consultation. Two of the position papers were submitted multiple times.
By way of summary, the key findings were:
Most respondents were ‘to a great extent’ or ‘to some extent’ aware of EU funding provided for the policy areas covered by the three programmes. 94% agreed that activities targeting gender-based violence and violence against children, young people and other groups were highly beneficial, followed by combating all forms of discrimination (91%), gender equality (89%), non-discrimination (89%), citizens’ rights and the rights of persons with disabilities (89%), children’s rights (89%), data protection (85%), democratic participation (83%), consumer rights (79%) and EU funding for activities on EU Remembrance 64%.Most respondents thought that the programmes contributed to the objectives of: citizens cooperating better with one another across borders (72%, 38 of 53); citizens feeling that they belong to the European Union (66%, 35 of 53); citizens being more aware of gender-based violence (72%, 38 of 53); promote a better understanding of EU citizens’ rights (73%, 39 of 53).
There were high levels of agreement that the three programmes had contributed to achieving their objectives, and that there was sufficient funding available from the programmes to support the types of activities in which respondents’ organisations were interested. This confirmed the effectiveness and efficiency of the programmes. Large proportions of respondents also indicated that the objectives of the programmes remained relevant – in the case of CERV, the objective concerning a more democratic EU, respect for the rule of law, fundamental rights, and democratic dialogue was unanimously deemed relevant.
If CERV funding were to cease, the potential consequences, as perceived by respondents, painted a concerning picture. A significant majority agreed that the civil space would shrink further or that progress towards gender equality would slow (88%), and that the sense of European citizenship and identity would decline (91%).
Overall, the position papers reflected a consensus on the importance of a coordinated, flexible, and inclusive approach to funding and supporting democratic values and rights within the CERV programme. The position papers generally shared a consensus on the importance of the CERV programme and its role in supporting civil society and fundamental rights. A summary report was published on the European Commission ‘Have your say’ webpage in August 2024.
4.10.Expert interviews
Six expert interviews were conducted pertaining to each of the four strands of the CERV programme. These interviews confirmed the conclusions and lessons learned as regards the relevance and EU added value of the programme.
Annex VI: State of play in the policy areas of CERV
Equality, rights, discrimination and violence
European societies continue to face challenges such as discrimination, violence, and inequality. The COVID-19 pandemic and Russia’s war of aggression against Ukraine have exacerbated these issues, particularly affecting vulnerable groups like children, people with disabilities, and women. The pandemic has led to increased domestic violence, digital inequality, and mental health concerns, while the war has resulted in a surge in trafficking and sexual violence against girls and young women entering EU Member States in need of protection. As revealed by a 2023 Special Eurobarometer, discrimination remains widespread, with 42% of respondents agreeing to this statement on the grounds of religion or belief, 45% based on age, 49% based on disability, and 54% regarding the ground of sexual orientation.
The pandemic also worsened gender equality in Europe, with employment for women reduced by 2.2 million across the EU during the first wave. Work-life balance conflicts were heightened, especially for women with young children aged 0-5 years, and online schooling represented a new form of unpaid care for parents, especially women, with a study by the European Institute for Gender Equality showing that mothers had to deal with interruptions by children more often than fathers while teleworking.
Structural inequalities between men and women still persist in the EU, as evidenced by the substantial gender gaps in employment (10.2p.p. in 2023), pay (12.7% in 2022) and pension retributions (25.4% (age 20-64) in 2023). Eurostat data show that, in the EU, care responsibilities keep some 7.7 million women out of the labour market, compared with just 450 000 men. This has, with negative consequences on female labour market participation, work and progression, pay and pensions and even in the autonomy of life-choices such as, in worst cases, their ability to terminate abusive relationships.
Racism – including anti-Roma, anti-black and anti-Muslim racism, antisemitism, LGBTIQ-phobia and other forms of intolerance are still deeply anchored in the European society. Overall, hate speech and hate crime have been steadily on the rise in the past decade, exacerbated by the spread of digital tools and social media.
Children are facing obstacles in the fulfilment of their rights, and recent crises have aggravated pre-existing gaps and challenges in child protection systems. Offline, 13% to 29% of children (15-years old) in different Member States reported being frequently bullied. 13% of 12-16-years-olds in the EU have received unwanted sexual requests online multiple times.
As shown in a 2023 Eurobarometer, 49% of respondents think that discrimination based on disability is widespread. While individuals’ awareness of their data protection rights has increased, an understanding of what this means in practice is still lacking.
33% of women in the EU have experienced physical and/or sexual violence. It is estimated that 2% of LGBTIQ+ persons in the EU have been actually submitted to ‘conversion practices’ and 5% have been offered conversion, although the real figures could be much higher.
EU citizenship, EU values and citizens’ participation
There are still significant proportions of EU citizens who do not identify as being European and do not feel like citizens of the EU.
Figure 31: In general, how much do you identify yourself as being European? (%)
Source: Special Eurobarometer on Values and Identities of EU citizens
Attitudes towards EU values – while generally positive across the EU – show large discrepancies among the EU-27 Member States.
Figure 32: What is your attitude to EU values? (EU-27, %)
Source: Special Eurobarometer on Values and Identities of EU citizens
There are still significant proportions of EU citizens unaware of the recent history that their country shares with other European countries, and we note pronounced differences between the EU-27 Member States in that regard. Disinformation and historical revisionism are ripe online, exploiting vulnerabilities and specific target groups within European societies. Moreover, Russia’s war of aggression against Ukraine has further highlighted how manipulation of historical events can be used as a tool for foreign interference to achieve strategic objectives. It underscored the need for preservation of historical accuracy, critical reflection on the past, and transmission of memory to future generations to counter hostile revisionist narratives and pre-empt a repetition of past mistakes.
While there are no recent data available on Holocaust awareness among EU citizens specifically, 2018 Eurobarometer data showed that 1 in 20 Europeans had never heard of the Holocaust, and only 43% of Europeans thought it was sufficiently taught in schools. A recent UNESCO report further highlighted the extent of Holocaust denial and distortion on social media, with highly concerning statistics: Holocaust denial or distortion was present on all online platforms and nearly half (49%) of all content on public Telegram channels that discussed the Holocaust either denied or distorted its history.
Citizens’ participation in political and social activities remains limited. Between 2021 and 2024, there was a notable drop in those who declared that they voted in local, national or European elections.
Figure 33: Did you participate in the voting for local, national or European elections? (EU-27) (%)
Source: Special Programme Eurobarometer 2021 & 2024
Moreover, participation in other aspects of political and civic life, such as getting involved in trade unions, political movements or parties, NGOs and CSOs, posting opinions on online social networks, obtaining information on current issues on online social networks, and taking part in volunteering activities or local community projects has remained low.
As regards awareness of their rights derived through EU citizenship, an analysis of Eurobarometer data shows that, in 2024, 39% of respondents did not know about their rights an EU citizen.
Figure 34: Are you aware of your rights as an EU citizen? (%)
Source: Eurobarometer European Citizenship - May 2024
As noted in the 2024 Eurobarometer report, around four in ten EU citizens are dissatisfied with the way democracy works in their country and in the EU, and one third feel that the EU’s core values, such as fundamental rights, democracy and the rule of law, were not well protected in their country.
Figure 35: On the whole, are you satisfied on how the democracy works in your country (EU-27) (2004-2024) (%)
Source: Eurobarometer
Women and young people also remain under-represented in political decision-making (the Fundamental Rights Report 2024). Lack of representation represents a significant missed opportunity and is not conducive to ensuring that fundamental rights violations are properly addressed, if and/when they occur.
Civil society
Moreover, the situation of civil society – whose meaningful participation in public affairs is crucial for the full implementation of core values of the EU, is uneven across the EU. In 12 Member States, it is classified as ‘narrow’, and in three Member States, as ‘obstructed’. Moreover, CSOs in the EU continued to face verbal attacks by politicians and third parties, and even government officials, as noted in the 2023 FRA update on ‘Protecting civil society’. Attacks by third parties and excessive state interference continue to threaten the civil society space, as do threats and direct attacks against journalists, which affect media freedom in some EU Member States.
Figure 36: Have you experienced any of the following? (EU-27) (%)
Source: FRA Civic space consultation in 2023
Table of abbreviations
|
Term or acronym
|
Meaning
|
|
ABAC
|
Accrual-Based Accounting
|
|
AGs
|
Action Grants
|
|
AMIF
|
Asylum, Migration and Integration Fund
|
|
CERV
|
Citizens, Equality, Rights and Values
|
|
CIV
|
Civil Society (projects)
|
|
CIV-OG
|
Civil Society Organisations and Think Tank Operating Grants
|
|
CSOs
|
Civil Society Organisations
|
|
DB
|
Budget discharge (décharge budgétaire)
|
|
DG BUDG
|
Directorate-General for Budget
|
|
DG CNECT
|
Directorate-General for Communication Networks, Content and Technology
|
|
DG COMM
|
Directorate-General for Communication
|
|
DG DIGIT
|
Directorate-General for Digital Services
|
|
DG EAC
|
Directorate-General for Education, Youth, Sport and Culture
|
|
DG EMPL
|
Directorate-General for Employment, Social Affairs and Inclusion
|
|
DG ENV
|
Directorate-General for Environment
|
|
DG ESTAT
|
Directorate-General Statistical Authority of the European Union
|
|
DG HOME
|
Directorate-General for Migration and Home Affairs
|
|
DG INTPA
|
Directorate-General for International Partnerships
|
|
DG JUST
|
Directorate-General for Justice & Consumers
|
|
DG NEAR
|
Directorate-General for European Neighbourhood and Enlargement Negotiations
|
|
DG REFORM
|
Directorate-General for Structural Reform Support
|
|
DG REGIO
|
Directorate-General for Regional and Urban Policy
|
|
DG RTD
|
Directorate-General for Research and Innovation
|
|
DG SANTE
|
Directorate-General for Health and Food Safety
|
|
EACEA
|
Education, Audiovisual and Culture Executive Agency
|
|
EC
|
European Commission
|
|
ECI
|
European Citizens’ Initiatives
|
|
EfC
|
Europe for Citizens
|
|
ESF
|
European Social Fund
|
|
EU
|
European Union
|
|
ISSG
|
Inter-Service Steering Group
|
|
LGBTIQ+
|
Lesbian, Gay, Bisexual, Transgender, Intersex and Queer (persons)
|
|
MFF
|
Multiannual Financial Framework
|
|
NC
|
National Coverage
|
|
NGOs
|
Non-governmental Organisations
|
|
NT
|
Networks of Towns
|
|
OGs
|
Operating Grants
|
|
RCHI
|
Promote the rights of child
|
|
RCIT
|
Promote the rights deriving by EU citizenship
|
|
RCON
|
Enforce consumer rights
|
|
RDAP
|
Prevent violence against children, young people, women, and other groups at risk
|
|
RDAT
|
Ensure the highest level of data protection
|
|
RDIB
|
Promote the right of persons with disabilities
|
|
RDIS
|
Promote non-discrimination
|
|
REC
|
Rights, Equality and Citizenship
|
|
REM
|
Remembrance (projects)
|
|
REM-OG
|
Remembrance Operating Grants
|
|
RGEN
|
Promote equality between women and men and gender mainstreaming
|
|
RRAC
|
Prevent and combat racism, xenophobia, homophobia and other forms of intolerance
|
|
RSB
|
Regulatory Scrutiny Board
|
|
SO
|
Specific Objectives
|
|
SWD
|
Staff Working Document
|
|
ToR
|
Terms of Reference
|
|
TT
|
Town Twinning
|
