52020SC0289

Annex 1: Procedural information

Annex 2: Stakeholder consultation

Annex 3: Intervention logic

Annex 4: Methods and analytical models

Annex 5: implementation status of the actions of the Strategy

Annex 6: Evaluation questions

Annex 7: Lists of figures and tables

Glossary

Term or acronym	Meaning or definition
DG	Directorate-General: is a policy department of the European Commission
DHLG	Disability High Level Group
EaSI Programme	Employment and Social Innovation Programme
EESC	European Economic and Social Committee
EMCO	Employment Committee
ERDF	European Regional Development Fund
ESF	European Social Fund
ESI Funds	European Structural and Investment Funds
EU	European Union
EU-LFS	EU Labour Force Survey
EU-SILC	EU Statistics on Income and Living Conditions
FRA	Fundamental Rights Agency
REC Programme	Rights, Equality and Citizenship Programme
SPC	Social Protection Committee
Strategy	European disability strategy 2010-2020
Study	Study supporting the evaluation of the European disability strategy (2010-2020)
UNCRPD	United Nations Convention on the Rights of Persons with Disabilities
AT	Austria
BE	Belgium
BG	Bulgaria
HR	Croatia
CY	Cyprus
CZ	Czechia
DK	Denmark
EE	Estonia
FI	Finland
FR	France
DE	Germany
EL	Greece
HU	Hungary
IE	Ireland
IT	Italy
LV	Latvia
LT	Lithuania
LU	Luxembourg
MT	Malta
NL	the Netherlands
PL	Poland
PT	Portugal
RO	Romania
SK	Slovakia
SI	Slovenia
ES	Spain
SE	Sweden
UK	the United Kingdom

1.Introduction

1.1 Purpose and scope

The purpose of this evaluation is to review the implementation of the European disability strategy 2010-2020 1 (the Strategy) and to provide conclusions that can be used as a basis for future policy development 2 .

The evaluation assesses to what extent the objectives of each area of the Strategy, detailed in section 2.1.2, have been achieved and to what extent the actions for achieving these objectives have been implemented. In addition, the evaluation highlights strong points and weaknesses in the implementation of the Strategy and provides overall conclusions.

The scope of the evaluation is largely focused on the actions undertaken by the EU institutions and the European Commission in particular, and only to a limited extent on actions undertaken at Member State level. This limitation has highlighted the need to address in the future also the progress at national level.

As the Strategy is the policy framework for implementing the United Nations Convention on the Rights of Persons with Disabilities (UNCRPD, the Convention) 3 at EU level, the evaluation has also assessed how the Strategy has influenced the implementation of the Convention at EU level.

The evaluation covers all five criteria set out by the Better Regulation requirements, namely effectiveness, efficiency, coherence, relevance and EU added value.

This document largely relies on an external supporting study 4 (the Study), and on input gathered via a broad stakeholder consultation process described in Annex 2.

The geographical scope of the evaluation is the EU in its pre-Brexit composition (28 Member States). The time span covered by the evaluation is November 2010 to December 2019. However, a few references to policy developments beyond this timeframe have been included in this Staff Working Document for the sake of completeness 5 .

The Strategy will end in 2020. In line with the "evaluate first" principle, which is part of the Better Regulation 6 regulatory policy, the results of the evaluation will be used as a basis for the formulation of the Commission’s future priorities and initiatives in the field of disability.

2.Background to the intervention

2.1 Description of the intervention and its objectives

2.1.1 Need for action

The Treaty on the Functioning of the European Union (TFEU) 7 requires the EU to combat discrimination based on disability when defining and implementing its policies and activities (Article 10) and it gives it power to adopt legislation to address such discrimination (Article 19). The Charter of Fundamental Rights of the European Union 8 prohibits discrimination on the basis of disability (Article 21) and recognises the right of persons with disabilities to independence, social and occupational integration and participation in the life of the community (Article 26).

Since 1983, the Commission has supported the development of a European disability policy through a succession of action programmes 9 . In 2003 the Commission launched a disability action plan for the period 2003-2010 10 . While important steps were taken in this period to improve the lives of persons with disabilities in the EU, the external contractor in charge of the action plan evaluation also reported critical views of stakeholders on the adequacy of the existing policy and regulatory frameworks at the EU level to address the needs of people with disabilities 11 .

Building on the results of the action plan, the European disability strategy was adopted in 2010, while the process for the ratification by the EU of the UNCRPD was underway. Through the ratification, the EU committed to comply with the Convention’s obligations and set up the necessary coordination and implementation mechanisms for its implementation.

The Strategy was conceived as the policy framework through which the EU would deliver the commitments made under the UNCRPD in line with the respective competences provided for in the Treaties.

2.1.2 The Strategy

The Communication 12 which set-out the Strategy in 2010 presented it as “a framework for action at EU level, as well as with national action to address the diverse situation of men, women and children with disabilities”.

The overall purpose of the Strategy was to “promote a barrier-free Europe and to empower persons with disabilities so that they can enjoy their rights and participate fully in society and economy”. Achieving those objectives required policies and actions at the level of the EU and of the Member States, with due regard to their respective competences. The Strategy identified eight key areas for improvement with the following specific objectives:

1)Accessibility: making goods and services accessible to persons with disabilities and promoting the market of assistive devices

2)Participation: ensuring that persons with disabilities enjoy all benefits of EU citizenship, removing administrative and attitudinal barriers to full and equal participation, promoting the provision of quality community-based services

3)Equality: eradicating discrimination on the grounds of disability in the EU

4)Employment: enabling many more persons with disabilities to earn their living on the open labour market

5)Education and training: promoting inclusive education and lifelong learning for pupils and students with disabilities

6)Social protection: promoting decent living conditions for persons with disabilities

7)Health: fostering equal access to health services and related facilities for persons with disabilities

8)External action: promoting the rights of persons with disabilities within the EU external action

The Strategy also included four instruments to support its overall implementation: awareness raising; financial support; statistics, data collection and monitoring; mechanisms required by the UNCRPD. The instruments, like the areas, pursue specific objectives:

a)Awareness raising: raising society’s awareness of disability issues and foster greater knowledge among persons with disabilities about their rights

b)Financial support: optimising the use of EU funding instruments for accessibility and non-discrimination and increasing visibility of disability-relevant funding possibilities in post-2013 programmes

c)Statistics, data collection and monitoring: supplementing the collection of periodic disability-related statistics with a view to monitoring the situation of persons with disabilities

d)Mechanisms required by the UNCRPD: establishing mechanisms at EU level for the implementation of the UNCRPD

An annex to the Strategy included the “list of actions” 13 to be implemented during the period 2010-2020 for achieving the Strategy’s objectives.

2.1.3 The UNCRPD

The Strategy is closely linked with the UNCRPD. The Commission adopted the Strategy in November 2010, shortly before the EU ratified the Convention in December 2010. By 2018, all EU Member States had also ratified the Convention.

The UNCRPD is a legally binding human rights treaty. Consequently, at the EU level, its provisions are an integral part of the EU’s legal order and all existing and future legislation must be in line with the UNCRPD. As parties to the Convention, both the EU and its Member States are obliged to implement their obligations under the Convention according to their respective competences.

However, the scope of the Convention is very broad and it covers all the aspects of the life of persons with disabilities for their full inclusion in society. The Strategy, through its eight areas and its four implementation instruments, covered part of the scope of the UNCRPD.

2.1.4 The competences of the EU

The EU has only the competences conferred on it by the Treaties and may only act within the limits of its competences 14 (principle of conferral).

Based on this principle, the EU and the Member States are obliged to implement their obligations under the UNCRPD according to their respective competences.

The Council Decision concerning the conclusion by the EU of the UNCRPD 15 declared the extent of EU competences with respect to the matters governed by the Convention 16 , and a secondary legislation was identified and listed in a declaration in annex to illustrate the extent of the area of EU competence. The Decision explains that the scope and the exercise of EU competence are, by their nature, subject to continuous development and that the EU will complete or amend the declaration, if necessary, in accordance with Article 44(1) of the UNCRPD.

This division is also reflected in the scope of the Strategy which includes fields where the EU and the Member States have “shared competences”, and fields where the EU has only a “supporting competence”.

In the fields falling under shared competences, both the EU and the Member States have the power to adopt legislative measures. Fields of shared competence broadly include internal market, transport, social policy, employment, equality, development cooperation and humanitarian aid.

In the fields where the EU has supporting competences, the EU can only supplement, support, and coordinate the actions of the Member States. Areas where the EU has supporting competences include health, education and training.

In order to comply with the principle of conferral and with the obligation for the EU to only act within the limits of its competences, the Strategy was issued in the form of a communication, so that its content, although binding for the Commission, did not directly commit other EU policy makers or the Member States. Member States were obliged to follow-up on the Strategy’s actions only when those actions have a binding nature. This was the case, for example, of the European Accessibility Act 17 , the Web-Accessibility Directive 18 , the Audiovisual Media Services Directive 19 , or the European Electronic Communications Code 20 , which were part of the Strategy’s implementation plan and, once adopted, are binding also for the Member States.

As a result, the evaluation focuses on the actions taken by the EU and the Commission in particular, in relation to the Strategy and, to some extent, on actions taken at Member State level to implement EU decisions. This limitation showed the need for a broader view about national action and progress in the future.

2.1.5 Intervention logic

In the years prior to the Strategy the proportion of persons with disabilities across the EU was about 16 % of the working-age population 21 , and there was general agreement that demographic trends (i.e. ageing population) would lead to an increase in the number of persons with disabilities 22 . Evidence showed that persons with disabilities were poorer than the average European, less likely to have a job, and they faced barriers in accessing goods and services such as education, healthcare, transport, housing and technology 23 .

The Strategy aimed to mobilise or establish several policy instruments and governance structures (inputs) for achieving its goals. This includes Commission staff time and financial resources to deliver specific actions.

Specific EU level actions were devised for each of the eight areas of action and the four implementation instruments (the actions are described in a “list of actions” 24 published in annex to the Strategy).

The Strategy intended to supplement and support Member State actions. In that regard, Member States contributed toward achieving the Strategy objectives through using national resources, such as national, regional, and local funds, in combination with the EU funds and by implementing policy initiatives at national level.

Implementation of specific actions of the Strategy can be observed in the form of outputs and results / impact. The key outputs entail the adoption and transposition of EU legislation on disability, the development of toolkits for the implementation of EU disability-related policies, the organisation of awareness-raising sessions and dialogue events with regard to disability rights, the pursuit of research studies on disability, and the overall mainstreaming of disability considerations in EU and national policies.

The expected results and impacts of the intervention were the gradual removal of disability barriers and the improvement of the situation of persons with disabilities in the specific areas of the Strategy. Impacts and results were therefore expected at an individual level, and the increase of the capacity of the Member States to improve the lives of persons with disabilities at a system level. The achieved outcomes from these interventions were expected to generate sustainable and lasting effects in the inclusion and participation of persons with disabilities, increasing their capacity to fully enjoy their fundamental rights and freedoms.

The intervention logic was reconstructed in the Study and its graphical presentation is included in Annex 3.

2.2 Baseline and points of comparison

The Strategy pursued its objectives through the implementation of a “list of actions” to be carried out and completed during the Strategy’s ten-year period. This list of actions serves as baseline for the Strategy. The extent to which the actions have been implemented is the main point of comparison to evaluate how the Strategy was implemented over time.

The Strategy did not have a clear method for qualitatively assessing the progress of each action. As a result, evaluating the extent to which actions were implemented has required the definition of a specific methodology, which is reported under Section 4 and Annex 4 of this Staff Working Document.

3.Implementation / state of Play

3.1 Description of the current situation

The abovementioned “list of actions” 25 constitutes the operational implementation plan across the Strategy’s thematic areas. The implementation plan comprises 150 measures – legislation and other related policy instruments such as soft law measures, standards and research/studies – and was initially intended for the first five years of the Strategy. The mid-term review of the Strategy 26 published in 2017 confirmed the relevance of the actions also for the remaining period of implementation.

In general, the findings of the supporting Study show that 101 actions of the Strategy (67% of total) were fully implemented throughout the whole implementation period. 40 actions (27% of total) were partially implemented and 9 actions (6% of total) were not implemented.

Figure 1: Implementation of the Strategy overall

Source: ICF analysis

Figures 2.1 and 2.2: Implementation by thematic area (number and percentage)

Source: ICF analysis

The areas with the highest level of implementation were Participation, Equality and Employment. The area with the lowest level of implementation was External action.

In relation to the implementation instruments, all the actions were fully or partially implemented. Only one action was not implemented for the implementation instrument of Statistics, data collection and monitoring (name of the action: “Work towards a legal basis for a health and social integration survey implementing Regulation (EC) 1338/2008 on public health statistics”).

Figures 3.1 and 3.2: Implementation by implementation instrument (number and percentage)

Source: ICF analysis

The detailed overview with the level of implementation of each action is reported in Annex 5 of the Staff Working Document.

4.Method

4.1 Short description of methodology

The evaluation of the Strategy is based on a set of evaluation questions linked to the five assessment criteria defined by the Better Regulation guidelines (effectiveness, efficiency, coherence, relevance and EU added value). These questions were broken down into sub-questions and developed as operational questions appropriate for stakeholder responses. An evaluation framework describing this process is included in the Study 27 . In addition, the analysis of the criteria built on the intervention logic is represented graphically in Annex 3 of this Staff Working Document.

In line with the Better Regulation provisions 28 , the evaluation has first to provide answers to each evaluation question and then provide conclusions about the overall implementation of the Strategy.

4.1.1 Sources of information

The Roadmap for the evaluation of the European Disability Strategy 2010-2020 29 was published on the European Commission ‘Have your say’ web portal 30 from 28 September 2018 until 26 October 2018.

An external and independent supporting study (“the Study”) was conducted to support the evaluation, between March and December 2019. The main objective of the Study was to evaluate the effectiveness, efficiency, relevance, coherence and EU added value of the Strategy and assess the extent to which the Strategy has reinforced the implementation of the UNCRPD. The methodological approach to the Study combined quantitative and qualitative data that have been systematically gathered and triangulated to assess the implementation of the Strategy and to formulate judgements for answering the evaluation questions and providing conclusions on the Strategy.

Evidence has been gathered during the whole evaluation process through different methods and from different sources:

-Desk research: analysis of existing studies, policy documents, position papers and legislative acts related to the Strategy and to EU disability policy.

-Targeted consultations: structured interviews and focus groups with 182 selected stakeholders at EU and at Member State level. An inclusive research methodology has been adopted to allow participation by the widest possible numbers of stakeholders.

-Online public consultation: an internet-based consultation (running from 31 July until 13 November 2019) open to all EU citizens and stakeholders was published in the Commission ‘Have your say’ web portal. 2,547 respondents took part in the consultation. 573 (22.5%) responses were from organisations, whereas 1,974 (77.5%) were from individuals.

-Online dedicated consultations: online surveys targeting organisations representing persons with disabilities and representatives of Ministries responsible for disability policy at national level.

-Public events: gathering of feedback from participants to EU high-level events dedicated to disability policy and to the evaluation of the Strategy.

Further information about the consultation work is included in Annex 2.

The information collected through these different methods has been analysed and compared in order to test its validity and internal consistency (triangulation). The results have been consolidated and combined with analytical techniques to detect possible trends and patterns.

The implementation plan of the Strategy did not include indicators or benchmarks for measuring the level of implementation of the actions or the level of achievement of its objectives. As a result, the evaluation of the Strategy required a method for determining to which extent the actions of the Strategy have been implemented and its objectives achieved.

Each action of the Strategy has been analysed and assessed against a scale of possible outcomes.

Actions have been assessed as:

-Fully implemented

-Partially implemented

-Not implemented

Explanations about the criteria for determining the outcomes are reported in Annex 5.

The extent to which objectives were achieved is strongly determined by the level of implementation of the underlying actions. However, also other elements were used for the assessment of the objectives such as the perceived relevance of the actions, the relative state of play in 2010 and in 2019 (when statistical data is available), the feedback from stakeholders, or the influence of decisions taken by other institutions at Member State or EU level. The resulting judgements have been used to formulate answers to the evaluation questions and to draw conclusions about the implementation of the Strategy.

4.2 Limitations and robustness of findings

Issues encountered during analysis:

-Statistical data, and notably disaggregated data, for assessing the impact of the Strategy on the situation of persons with disabilities is not always available or updated. Nevertheless, data disaggregated by age and gender has been available on an annual basis in relation to prevalence of disability, employment, education and poverty risk rates. Health-related data has been available, also disaggregated by age and gender every several years. Where conclusions on the impact of the Strategy could not be fully supported through quantitative information, they were discussed with relevant stakeholders, including non-governmental organisations and Member States representatives, and submitted to expert advice, namely through the contractor’s network of experts, to test their robustness and reliability;

-The absence of specific indicators and benchmarks for assessing the Strategy’s implementation and achievements in relation to the objectives made it necessary to develop an ex-post method for measuring the progress of the Strategy. However, there are indicators published annually on the gaps between persons with and without disabilities, in relation to the three headline targets of Europe 2020;

-Some of the effects of the Strategy’s actions are to materialise only after the end of the Strategy;

-Improvement in the situation of persons with disabilities can result both from actions at EU-level and at national and regional level. As the evaluation of the Strategy focused on EU-level actions, the assessment of the effectiveness of the Strategy in improving the situation of persons with disabilities cannot be directly correlated. Therefore, an estimation was provided, mainly based on the extent to which the actions of the Strategy were implemented. This assessment was also integrated with expert advice, feedback received by relevant stakeholders, and, where available, analysis of statistical data;

-In relation to efficiency, the Study made efforts to estimate the costs of the Strategy and to quantify its results. However, such an estimation is difficult to assess in objective terms. Therefore, conclusions about the efficiency of the Strategy also relied on elements such as professional judgement, expert advice and stakeholder feedback.

Despite these limitations, the analytical framework and the data collected were adequate with regard to quality and breadth of representation from different categories of stakeholders. Where available data was limited, the evaluation extended its research base. Where shortcomings could not be mitigated fully, the Study relied on the contractor’s professional judgment, expertise and stakeholders’ feedback as the basis for providing assessment and conclusions. At the end of the evaluation exercise, this allowed for methodologically robust findings and conclusions. More details on the methodology adopted and on how issues were mitigated for each analytical task are reported in Annex 4.

5.Analysis and answers to the evaluation questions

5.1 Effectiveness

The conclusions about the effectiveness of the Strategy and about the extent to which its overall objective, as well as objectives per key areas and implementation instruments have been achieved rely mainly on: the extent to which actions in the relevant areas have been implemented; expert advice; feedback received by relevant stakeholders; and, where available, analysis of statistical data.

Overall, the Study suggests there has been good progress since 2010 in the participation of persons with disabilities in society and economic life, in the promotion and protection of their rights, and in the implementation of the UNCRPD across the EU.

56% of respondents to the public consultation carried out during the evaluation, agreed or somewhat agreed that the situation for persons with disabilities improved between 2010 and 2020 in their Member State (31% somewhat disagreed or disagreed). 46% of all respondents agreed or somewhat agreed that the situation for persons with disabilities improved over this period across the EU (11% somewhat disagreed or disagreed).

The targeted stakeholder consultations in the Member States show that in the large majority of countries, the situation of persons of disabilities is seen as having improved at least to some extent over the last ten years.

Table 1: Views on whether the situation for persons with disabilities improved in the Member States over the last ten years

Agree

Somewhat agree

Neither agree nor disagree

Somewhat disagree

Disagree

DE, DK, EE (3)

AT, BE, CY, CZ, ES, FI, FR, EL, HR, IE, LT, LV, MT, PT, SK (15)

BG, HU, IT, LU, NL, RO, SE, SI (8)

PL (1)

UK (1)

Note: a) The Study national experts’ assessment is based on the information gathered from the stakeholder interviews and focus groups carried out in Member States. b) The table indicates the most common selection across all areas of action for each country; where there was no consensus among stakeholders, ‘neither agree nor disagree’ was selected.

Source: Member State analysis performed by national experts in the context of the Study

There is less agreement on the extent to which the Strategy helped to improve the situation for persons with disabilities. The Study’s national experts in seven Member States agreed that the Strategy has helped (BE, DE, EE, NL, PT, CZ, RO). But in ten Member States they disagreed (BG, ES, FI, SI, HU, IE, LT, MT, PL, SE) and in eleven Member States the experts expressed neither the agreement nor disagreement (AT, CY, DK, FR, EL, HR, IT, LU, LV, SK, UK).

On the other hand, many EU-level stakeholders consulted during the Study, and especially the umbrella organisations representing persons with disabilities at EU level, agree that the existence of such a strategy at EU level is in itself crucial, as the adoption of a high-level EU policy document explicitly dedicated to disability has helped placing the disability agenda higher among the EU policy priorities.

In terms of implementation, the Study concludes that not all actions were fully implemented, and that they contributed to a different extent to the achievement of their objectives.

For example, in the area of Equality, almost all the actions were fully implemented. But the Study considered its objective as partially achieved because one of its most important actions (“Support the negotiation in Council of the draft Directive on equal treatment beyond the field of employment”) did not lead to the adoption of the Directive by the Council.

Another element that limited the achievement of the Strategy is the broad scope of its overall objective. The Study suggests that for ensuring full achievement the Strategy would have required a wider and more comprehensive set of instruments and actions than those foreseen 31 , including actions undertaken by Member States.

The following sections present in more detail the implementation of the Strategy across each of its areas and implementing instruments. Additional sections are dedicated to mainstreaming, to the identification of success as well as of hindering factors and to stakeholders’ engagement in the implementation of the Strategy.

5.1.1 Achievement of objectives and implementation the actions in eight key areas

5.1.1.1 Accessibility

There has been on overall improvement in this policy area of accessibility whose objective was to ensure the accessibility of goods, services, including public services and assistive devices for persons with disabilities. 63% of actions in this area were fully implemented, including through legislation. The Study also suggests that the progress has been uneven across areas and that accessibility for persons with disabilities has been partially ensured.

Accessibility is the area where the Strategy delivered its most visible legislative outputs, which reflect the level of competences of the EU: the adoption of the European Accessibility Act 32 and of the Web Accessibility Directive 33 , the adoption of the waterborne and bus and coach passenger rights Regulations 34 , the provisional agreement of the Council and the Transport and Tourism Committee of the European Parliament on the proposal of re-cast of the rail passengers’ rights Regulation 35 , the revision of the Commission Regulation on the technical specifications for interoperability relating to accessibility of the Union's rail system 36 , the inclusion of accessibility provisions in the Audiovisual Media Services Directive 37 , the new European Electronic Communications Code 38 , the so-called “Marrakesh” Directive 39 , and the inclusion of mandatory accessibility requirements for projects financed through EU funds. Regarding the problems related to compensation claims concerning damages of mobility equipment when travelling, this is addressed now in all passenger rights Regulations except for air passenger rights, where the Commission has proposed to amend the rules accordingly 40 .

The European Accessibility Act was a much-anticipated legislative action of the EU to ensure that goods and services including public services and assistive devices for people with disabilities are accessible, and it took four years to adopt it. All stakeholders considered its adoption as an important achievement of the Strategy and a positive step for improving the level of accessibility throughout the Union. Some stakeholders are critical about the voluntary approach to the application of the accessibility requirements for the built environment and see this as reducing the potential positive impact of the Act on the accessibility of goods and services by persons with disabilities.

The limited progress in the field of built environment remains a challenge. This includes the scope limitations of the Accessibility Act (accessibility of the built environment is subject to voluntary adoption by the Member States) and the fact that most of actions of the Strategy related to the built environment (standardisation mandate on accessibility of the built environment, inclusion of accessibility in curricula of architect and engineers, inclusion of accessibility in the Lead Market Initiative for sustainable buildings) were not implemented within the timespan covered by the evaluation 41 .

In the field of copyright, the “Marrakesh Directive” provides for a mandatory exception to copyright as regards printed works in formats accessible to persons with disabilities. As required by its Article 9, the Commission has started the preparation of a report on disabilities and on type of works which are not covered by the Directive.

The EU level targeted consultations held within the evaluation (interviews with the academic network of experts, international non-profit associations, and independent network organisations) highlighted that actions in the area of accessibility tend to focus on visible disabilities. For non-visible disabilities, such as intellectual disabilities and psychosocial disabilities in general, many of the accessibility needs have not been fully addressed by the Strategy (simplified environments, pace and complexity of information provided to the public, mental health literacy of public officials, specific awareness raising initiatives dedicated to invisible disabilities).

5.1.1.2 Participation

The Strategy’s actions were directed at enabling persons with disabilities to enjoy all the benefits of citizenship, removing administrative and attitudinal barriers to full and equal participation and providing quality community-based services, including access to personal assistance. More than 80% of these actions have been fully implemented.

The initiatives that most contributed to fulfilling the objective of this area are the following:

-Activities in support of independent living, and notably EU funding legislation 42 allowing the Member States to support measures for the shift from institutional to community-based care 43 , which led to concrete results in different Member States.

-Implementation of the European disability card 44 pilot project, conceived to address the issue of lack of mutual recognition in disability status among Member States and to ensure equal access to benefits across borders for people with disabilities, mainly in the areas of culture, leisure, sport and transport. A pilot of the card was launched in February 2016 in a group of eight EU Member States 45 By the end of 2020 the Commission will conclude the assessment on the implementation of this pilot project.

-The Inclusion and Diversity Strategy 46 which promotes participation of young people with disabilities into Erasmus+ youth mobility projects.

-Initiatives to promote participation into sports through funding for disability-specific sporting events, projects (through Erasmus+ 47 ), awareness raising actions (like the BeInclusive EU sport award 48 ) or events (such as the European Day of Persons with Disabilities conference 2019 49 where challenges encountered by persons with disabilities in sport were thoroughly discussed). In this area it is also important to mention the June 2019 Council Conclusions on Access to sport for persons with disabilities 50 .

The majority of stakeholders consulted in the evaluation process considered that the Strategy helped improve participation of persons with disabilities in society. They also considered that the Strategy made an important contribution in supporting the process of de-institutionalisation via the EU funds, thus supporting persons with disabilities to live independently and with adequate assistance.

However, the Study also suggested that the objective to achieve full participation of persons with disabilities in society was partially achieved, because, despite progress in this area, barriers to participation still persist. Among the main obstacles, the Study identified the following:

-Stakeholders from organisations representing persons with intellectual disabilities and charities have expressed concerns about the limited strategic vision in Member States for the transition from institutional to community living, and specifically, the lack of assessment of the needs of persons with disabilities and of measures to ensure their social inclusion. These concerns point out that the inclusion of the ex-ante conditionalities in the context of ESI Funds may not always be sufficient to guarantee the swift transition from institutional to community-based services. Nevertheless, stakeholders acknowledge the overall benefits deriving from the ex-ante conditionalities as well as the EU efforts to ensure their application. This resulted in better funding for projects that contribute to the social inclusion of persons with disabilities.

-There is still limited data collection on life conditions in residential institutions, and limited promotion of best practices on independent living.

-The Strategy has limited focus on autonomy of persons with disabilities. The respect of individual autonomy is one of the general principles of the UNCRPD and it refers to the right of persons with disabilities to have reasonable life choices, to be subject to minimum interference in private life and to be legally entitled to make own decisions (including political participation), with adequate support if required. The Study points out that autonomy was not part of the Strategy’s implementation efforts despite its importance for the overall participation objective. Although legislation on legal capacity is a national competence, the EU could have done more to promote the understanding of autonomy issues by public officers, especially among the judiciary and medical professionals.

-Organisations of persons with disabilities advocating for independent living underline that there is limited attention at EU level about the promotion of personal assistance through ESI Funds. They consider that funding personal assistance would be effective for improving overall participation of persons with disabilities in society.

5.1.1.3 Equality

The Strategy’s objective in the area of equality was to eradicate discrimination on the ground of disability across the EU. The Study suggests that this objective, although highly desirable, is an ambitious target to be achieved during the ten year implementation period of the Strategy.

The fulfilment of this objective would have required heavy changes (in terms of legislation, attitudes, social and economic conditions also at national and local level) that would have gone beyond the Strategy’s possible scope for achievement.

In terms of implementation, the Strategy proposed the adoption of eleven actions in the field of equality, and over 80% of the actions have been fully implemented (more details are available in Annex 5).

The shared competences of the EU in this field make it possible to adopt legally binding acts. One of the key actions under this area was “Monitoring the application and impact of Directive 2000/78/EC 51 for improving employment of persons with disabilities”. The action was fully implemented. In 2014, the Commission presented an implementation report 52 concluding that all Member States had taken the necessary measures to transpose the Directive into their legal orders. The report also highlighted that some Member States had faced initial difficulties in correctly transposing the provisions on reasonable accommodation. In its conclusions, the report highlights the importance of awareness raising to increase knowledge of the already existing protection and to ensure better practical implementation and application of the Directive.

Another key action under this area was to “Support the negotiation in Council of the draft Directive on equal treatment beyond the field of employment”. In 2008, the Commission presented a proposal 53 for a horizontal Directive on equal treatment (Equal Treatment Directive), but the proposal has since then been blocked in the Council because the required unanimity has not been reached so far. As a result, despite being partially implemented, the action did not lead to the expected results and the non-adoption of the Equal Treatment Directive has left a legal gap in terms of protection from disability-based discrimination.

The Equal Treatment Directive

The Commission adopted the proposal for an Equal Treatment Directive in 2008, which has not yet been adopted by the Council of the European Union, composed of all EU Member States.

The proposed Directive would expand protection throughout the European Union against discrimination on the grounds of religion or belief, disability, age and sexual orientation to the areas of social protection including healthcare, education, and access to and supply of goods and services including housing.

Many stakeholders consider that the adoption of this Directive is essential for prohibiting discrimination on the ground of disability.

The remaining actions under this area focused on awareness raising and they were fully implemented. However, the Study suggests that these measures were not sufficient for eradicating discrimination against persons with disabilities.

In relation to the area of equality, the Study suggests that the ambitious formulation of its objective, the non-adoption of the Equal Treatment Directive and the limited scope of the remaining actions, led to partial achievements in eradicating discrimination based on disability in the EU.

5.1.1.4 Employment

The Study suggests that the Strategy has been effective in implementing employment policy tools, but that its impact in increasing the participation of persons with disabilities in the open labour market is not clearly measurable.

In the area of employment 75% of actions were fully implemented, 19% partially implemented, and 6% not implemented. The Study suggests that the actions of the Strategy in the area of employment have been effective in relation to the following issues:

-Use of EU funds for supporting employment of persons with disabilities.

-Awareness raising about disability rights in the field of employment and specifically in the private sector.

-Putting in place diversity charters and exchanges with employers on diversity.

-Improving the statistical data on the situation of persons with disabilities in the field of employment.

In relation to policy tools, the study highlights that the European Pillar of Social Rights 54 (“the Pillar”), in its principle 17 “Inclusion of persons with disabilities”, explicitly asserts the right to services that enable persons with disabilities to participate in the labour market and in society, and to a work environment adapted to their needs.

In relation to statistical data, the Study highlights that the creation of the 2011 Labour Force Survey 55 ad hoc module focusing on employment of disabled people 56 and the imminent inclusion of a proxy variable on disability every two years in all population surveys coordinated by Eurostat are important achievements from a policy point of view. These results, combined with the analytical work of Academic Network of European Disability experts (ANED) 57 , are particularly important tools for the formulation of adequate policies and legislative initiatives in relation to disability and employment. On the other hand, the non-implementation of a specific indicator on employment of vulnerable groups by the EMCO indicators’ group appears as a major shortcoming in the evaluation.

Despite these results, the stakeholders’ views on the extent to which the Strategy enabled many more persons with disabilities to earn their living on the open labour market are significantly diverging. Organisations representing persons with disabilities, business organisations and citizens responding to the public consultation tended to consider employment among the worst performing areas of the Strategy due to the persisting gaps between persons with disabilities and without disabilities: only 40% of the respondents with knowledge of the Strategy to the public consultation agreed/somewhat agreed that progress for persons with disabilities had been made in the area of employment. On the other hand, institutional stakeholders at EU and Member State level consider that the Strategy has brought positive effects in the field of employment thanks to the improvements in statistical data and awareness raising.

In terms of general outcomes, there has been an increase in economic activity rates and employment rates for persons with disabilities since the adoption of the Strategy in 2010. The activity rate (age group 20-64) in the EU-28 increased from 56.1% (2010) to 62.4% (2018) and the employment rate for the same age group increased from 46% (2010) to 52% (2018). A similar trend is observed for persons without disabilities. In addition, the employment gap between people with and without disabilities has decreased only slightly over the years, from 26 percentage points in 2010 to 24.2 percentage points in 2018 in the EU 28. 58 Thus, the Study suggests that the actions of the Strategy, which complement the activities of the Member States, have not been decisive for significantly reducing the labour market participation gap.

It should also be noted that the respondents to the public consultation have identified employment as one of the five most important topics to be considered for the future policy (together with accessibility of buildings and public services, ageing and disability, independent living and inclusion in the community, children with disabilities and their families).

5.1.1.5 Education and training

The study suggests that, although the Strategy contributed to promote inclusive education and lifelong learning for pupils and students with disabilities, its impact in this area is not clearly measurable.

In this area, the EU has supporting competences and no power to propose or enact fundamental changes through legislation. Therefore, the activities proposed by the Strategy were mostly focused on soft law instruments, funding, and awareness-raising (research data collection and dissemination). The actions related to funding were those with highest implementation rate, whereas those relating to awareness raising were partially implemented.

In the area of Education and training 36% of actions were fully implemented, 55% were partially implemented, and 9% were not implemented. This makes Education and training one of the areas with lower implementation rates, with concerns over its effectiveness raised by the Study.

There was consensus among different types of EU level stakeholders interviewed that the Strategy achieved some significant results in supporting programmes for the inclusion of children with disabilities into education, trainings for teachers, educators and staff who work with children with disabilities, as well as in raising awareness and promoting good practices. Such programmes and policies supported the actions of the Member States to implement the Strategy objectives. Stakeholders representing persons with disabilities pointed out that in relation to education the Strategy did not include specific measures for the assistance to persons with disabilities in the transition from education to the labour market.

On the other hand, the Study highlighted that in terms of outcomes, educational gaps between persons with disabilities and persons without disabilities still persist. The share of early school leavers with a disability (age group 18-24) in the EU increased between 2010 and 2016 from 21.6% to 23.6%, with a decrease to 19.9% in 2018. Over the same period, the share of early school leavers with no disability fell (from 12% in 2010 to 11% in 2016 and 9.9% in 2018). The gap between persons with and without disabilities has not changed significantly over the years, slightly deteriorating from 9.6 percentage points in 2010 to 10 percentage points in 2018. 59

Data from ANED show that educational gaps also persist among persons having completed a tertiary or equivalent education. In 2010, 22.6% of persons with disabilities as compared to 34.7% of those without disabilities (age group 30-39) completed a tertiary or equivalent education. In 2018, 32.6% of persons with disabilities completed tertiary or equivalent education as compared to 43.2% of those without disabilities. Thus, the gap between the two groups has not changed significantly throughout the years, remaining at around 12 percentage points.

5.1.1.6 Social protection

For the area of social protection, the objective of the Strategy was to promote decent living conditions for persons with disabilities. The Study suggests that, although living conditions of persons with disabilities are a predominant competence of the Member States, and despite persisting gaps in social protection in relation to disability, the EU action in this field has contributed to better living conditions of persons with disabilities. Its policy actions, including the European Pillar of Social Rights, have contributed to the promotion of more inclusive social protection systems.

The organisation and financing of social protection systems is mainly a responsibility of Member States and governance tools at EU level are predominantly of a soft nature. Within the scope of its competence, the EU has carried out an important range of initiatives for the promotion of decent living conditions.

In terms of implementation, 67% of the actions under this area were fully implemented and 33% were partially implemented.

As concerns general outcomes in the area of poverty and social exclusion, the situation of persons with disabilities has not changed substantially over the years. The gap between persons with disabilities and persons without disabilities has widened. According to Eurostat EU SILC statistics, the rate of persons with disabilities at risk of poverty and social exclusion in the EU-28 was 29.6% in 2010 (20.8% for those without disabilities) and 29.1% in 2019 (18.4% for those without disabilities) 60 . This means that over the same period, while the risk of poverty and social exclusion decreased by 2.4 percentage points for persons without disabilities, it decreased by only 0.5 percentage points for persons with disabilities. The data also show that gaps between persons without disabilities and persons with disabilities as regards the risk of poverty and social exclusion persist (8.8 percentage points in 2010 and 10.7 percentage points in 2019).

Opinions of stakeholders about the success of the Strategy in promoting decent living conditions are diverging, even within the main groups of stakeholders. Overall, the Study suggests that the Strategy’s objective of this area has been partially achieved.

In 2017, the European Parliament, the Council and the European Commission jointly proclaimed 61 the European Pillar of Social Rights, an overall policy framework meant to achieve better working and living conditions in the EU. The European Pillar of Social Rights allows for a better degree of coordination, integration and improvement of social policies across the EU, and it specifically recognises the right of persons with disabilities to inclusion.

The European Commission has proposed legislation to advance with the implementation of the European Pillar of Social Rights, including a Directive on work-life balance 62 that was meanwhile adopted by the EU legislator, setting new and improving existing leave rights for parents and carers, and a Council recommendation on access to social protection for workers and the self-employed 63 that also covers disability and invalidity benefits.

As concerns financial support, the EU invested into social inclusion and related national reforms via its Funds and programmes, such as the European Social Fund (ESF and ESF+) 64 , the European Regional Development Fund, the Rights, Equality and Citizenship programme 65 and the EU programme for Employment and Social Innovation 66 , as well as through the Structural Reform Support programme 67 , where persons with disabilities are amongst the target groups. These investments helped to improve the situation of persons with disabilities, as they contributed to the development of social services, community based services and social economy. These improvements are also closely linked to deinstitutionalisation, a process that requires to couple the planned closure of long-stay residential institutions with the development of individualised services, and with making general services available to persons with disabilities.

5.1.1.7 Health

Health is mainly a national competence and the EU can only complement Member State action through initiatives such as policy monitoring, mutual learning and funding.

The Strategy included twelve actions to foster equal access to health services and related facilities for persons with disabilities. Throughout the period 58% of the actions under this area were fully implemented, 33% were partially implemented and 8% were not implemented.

Despite these results, the Study suggests that the Strategy did not fully achieve its objectives in this area. This is mainly due to the limited scope of the actions, the limited competence of the EU in this field, the limited availability of data and information on the situation of persons with disabilities and the lack of disaggregated data, which have prevented the development of appropriate policies and their monitoring.

Among the respondents to the public consultation only 38% of those with a disability agreed or somewhat agreed that the Strategy helped to improve health for persons with disabilities. Consultations with the Study national experts reach similar conclusions, with fifteen Member State reports concluding that the Strategy did not have a significant impact on improving the situation of persons with disabilities in the field of health at Member State level.

Despite these considerations, the Study suggests that the Strategy helped in mainstreaming disability considerations in legislation notably on cross-border healthcare (the Patients’ rights Directive 68 ). The Strategy also supported the production of statistics, data collection and monitoring activities, notably:

-The development of the 2011 EU-LFS 69 ad-hoc module on the employment of persons with disabilities 70

-The European Health Interview Survey 71

-The 2012/2013 European Health and Social Integration Survey (dedicated disability survey) 72

Data from EU-SILC indicate that there has been some improvement with regard to self-reported unmet needs for medical examination for persons with disabilities, which decreased from 7.6% in 2012 to 4.5% in 2019. As concerns persons without disabilities, it was 1.9% in 2012 and 1.1% in 2019. 73

The study – in particular the Member States analysis – also points to the importance of EU funding to resource implementation, particularly in areas like health where the EU has limited legislative competence.

5.1.1.8 External action

Overall, the actions implemented during the Strategy contributed to promote the rights of persons with disabilities within the EU external action. However, because only two of the eleven actions of the Strategy were fully implemented (all other actions were partially implemented), the evaluation suggests that the objective has been partially achieved. This seems also reflected in the results of the public consultation responses in relation to external action, which may reflect a low level of knowledge among respondents of the actions in this area of the Strategy.

At international level, the inclusion of external action as an area of the Strategy, together with the ratification by the EU of the UN Convention, has raised awareness that the EU wants to engage with countries across the globe to improve the situation for persons with disabilities.

There are several important actions of significance for the implementation of this thematic area, including the promotion of disability rights in the EU Action Plan on Human Rights and Democracy 2015-2019 74 and the publication by the European Commission of an operational guidance on the inclusion of persons with disabilities in EU-funded humanitarian aid operation 75 . Disability considerations have been included in the 2017 European Consensus on Development 76 as well as in the 2014 Council conclusions on a rights-based approach to development cooperation, encompassing all human rights 77 . In terms of capacity building, the network of disability focal points in EU Delegations and Headquarters has been relaunched in October 2019, and information on disability matters has been integrated in the Rights Based Approach training addressed to the staff.

The Study suggests that the EU initiatives in the field of external action follow the framework of the UNCRPD. However, the Study also suggests that there is limited evidence that EU initiatives undertaken in this field have been significantly influenced by the Strategy.

A shortcoming in the field of external action is related to the lack of indicators for planning and monitoring interventions with specific reference to disability (disability markers). The absence of these markers has limited the capacity to quantify disability initiatives in external action, and the extent to which the rights of persons with disabilities were promoted. However, the new OECD transversal disability policy marker 78 , which is in use in the Commission services of development cooperation since January 2019, will allow to follow up and assess the disability inclusion in the future.

Finally, a few stakeholders active in the field of international cooperation suggested that inclusion in the Strategy of further actions could have contributed to the achievement of the objectives of the Strategy to better reflect the EU’s commitment to disability inclusion. Further actions could have included, for example, promoting the use of the Washington Short Set of Questions 79 by partner countries as well as the refurbishing of EU institution buildings including EU Delegations in partner countries to make them accessible.

5.1.2 Achievement of objectives and implementation of the actions for the implementation instruments

5.1.2.1 Awareness raising

The objective of the Strategy in this field was to raise society’s awareness of disability issues and foster greater knowledge among people with disabilities of their rights and how to exercise them.

Out of the five actions included in the Strategy, four have been fully implemented and one has been partially implemented. Although further efforts are needed to promote positive and realistic images of persons with disabilities in all sectors of society, the Strategy has given a significant contribution to awareness raising, especially at EU level. The Study suggests that the Strategy has been instrumental in increasing the level of awareness about disability issues at EU institutional level, also leading to more inclusive EU policies and legislation.

The two main public disability events organised by the European Commission, the Access City Award 80 and the European Day of Persons with Disabilities 81 have gained growing visibility and momentum among disability stakeholders.

For the 2020 edition of the Access City Award 82 , 47 cities from across the EU sent applications. During its ten years of existence, 321 cities have participated in the Access City Award. However, only 33% of respondents to the public consultation declared having knowledge about the European Access City Award (30% of individual respondents and 56% of organisations). This suggests that knowledge about the Access City Award is common among institutional stakeholders, and not only at EU-level, but still limited among the general public.

The European day of Persons with Disabilities is an annual international conference gathering over 400 participants from all over the EU to discuss specific themes of importance for persons with disabilities. The conference is an important forum for discussion at EU level. It is regularly attended by Members of the College of the Commission, members of the European Parliament and representatives of EU and international institutions (such as the Fundamental Rights Agency and the United Nations), as well as umbrella organisations representing persons with disabilities at EU level, and persons with disabilities themselves, invited by the Commission.

In the public consultation, 65% of respondents declared having knowledge of the European Day of Persons with Disabilities (59% of individual respondents and 85% of organisations).

Major awareness raising campaigns have been organised throughout the European Union about topics related to disability. An example is the #EUvsDiscrimination campaign 83 on combating discrimination in the workplace. One of the main strands of the campaign addressed reasonable accommodation in the workplace for persons with disabilities (videos were produced and several events were held in Member States where the employment gaps between persons with and without disabilities are the largest).

Based on feedback from experts and stakeholders, the Study suggests that there is growing awareness in society about disability and about rights of persons with disabilities and that this is strongly related to the adoption by all EU Member States of the UNCRPD. The Study also suggests that in the area of awareness raising, the Strategy has been complementary to the UNCRPD and has contributed to its implementation.

5.1.2.2 Financial support

The objective of the Strategy in this field was to optimise use of EU funding instruments for accessibility and non-discrimination and increase visibility of disability-relevant funding possibilities in post-2013 programmes.

The Study suggests that the objective has been partially achieved.

The Strategy proposed the adoption of eleven actions in the field of financial support. Over 80% of the actions have been fully implemented. All stakeholders agree on the positive effect of EU funding for the implementation of Strategy, which in some key areas such as deinstitutionalisation was essential for its progress.

The Study also pointed out some weaknesses of the EU financial support system. Funding was limited and not adequate for the full implementation of the Strategy. Financial resources allocated to social issues usually did not include dedicated budget lines for disability. Stakeholders also pointed out that funding is not fully accessible for persons with disabilities due to inaccessible procedures and administrative burdens, and that there is insufficient involvement of the disability community in funding programmes.

5.1.2.3 Statistics, Data Collection and Monitoring

The objective of the Strategy in this field was to supplement the collection of periodic disability-related statistics with a view to monitoring the situation of persons with disabilities. The Study suggests that the objective has been partially achieved.

The Strategy proposed the adoption of thirteen actions. Over 80% of the actions proposed in this field by Strategy have been fully implemented.

Prior to the development of EU-wide statistics on disability, national definitions varied, as they were primarily used for determining benefit entitlements or for specific policies.

The Study highlights that the work of Eurostat on incorporating disability considerations in their data collection especially regarding the incorporation in 2021 of a 2-year proxy variable on disability in all population surveys coordinated by Eurostat 84 is a significant step towards a more effective data collection on the situation of persons with disabilities in the EU. In addition, the work of the Fundamental Rights Agency on indicators on the types and characteristics of institutional and community-based services in the EU could provide valuable information on how to improve community-based living.

However, in relation to overall indicator development and data collection, the Study suggests that there is a need for better data collection across the key priority areas of the Strategy and that statistics and data collection relating to the Strategy are insufficient, especially in relation to disaggregated data. This opinion has been expressed by several umbrella organisations representing persons with disabilities at EU level.

While several important steps were taken to improve the measuring of the situation of persons with disabilities in the EU, there are still challenges in this field that have not been addressed by the Strategy. The Study suggests that the main challenges are: 1) No concrete targets for the Member States or indicators for measuring their progress when implementing disability-related policies within the European Semester; 2) No single uniform approach to measuring access to healthcare across the EU; 3) No disaggregated data on disability.

5.1.2.4 Mechanisms required by the UNCRPD

The objective of the Strategy in this field is to establish a governance framework in line with Article 33 85 of the UN Convention on implementation and monitoring. The Study suggests that the objective has been achieved.

The Strategy proposed the adoption of four actions in this field, which have been fully implemented. As a result, a mechanism for the coordination and the monitoring of the implementation of the UNCRPD has been established at EU institutional level.

However, the Study also highlights that the mechanism does not include focal points at Directorate-General level within the European Commission as well as in other EU institutions. This somehow hindered the development of disability-related knowledge and the mainstreaming of disability considerations in EU legislative or policy actions.

5.1.3 Mainstreaming of disability issues in EU policy and legislation

The Study shows that, overall, the Strategy achieved positive results in mainstreaming disability issues in EU policy and legislation and that one of the main achievements of the Strategy was the placement of the disability agenda higher among the EU policy priorities.

At a strategic level, key EU policy initiatives such as the Europe 2020 strategy 86 , the European Pillar of Social Rights 87 , or the European Semester process 88 refer to disability issues and contain significant commitments to address the needs of persons with disabilities.

Disability issues are also reflected in other strategies, such as the EU Health Strategy 2008-2013 89 , the EU Digital Strategy 90 , the mid-term review of the Education and Training 2020 (ET2020) framework 91 , the Roadmap to a Single European Transport Area 92 , the EU Youth Strategy 93 , the EU’s Strategic Engagement for Gender Equality 94 and the 2020 Communication re-launching the European Research Area 95 , as well as the Communication on Achieving the European Education Area by 2025 96 and the Digital Education Action Plan 97 . Recent policy documents such as the EU Gender Equality Strategy 2020-2025 98 and the EU Action Plan against racism 2020-2025 99 point to the need to address intersectionalities between disability and other grounds of discrimination.

Importantly, the Strategy was also instrumental in mainstreaming disability into the European Semester and several Member States received recommendations explicitly targeting disability.

At a legislative level, the Strategy promoted greater change in areas where the EU has more competences. Initiatives with disability-related provisions include the European Accessibility Act, the passengers’ rights regulations and the adoption of legislation in EU investment and structural funds regulation stating the obligation that services and infrastructure financed through EU funds should be accessible to persons with disabilities.

Nevertheless, the Study also concludes that there are areas of improvement, particularly with regard to the mainstreaming of disability considerations in the field of consumer protection. The Study also suggests that, despite improved mainstreaming of disability in policy and legislation, overall awareness about the UNCRPD within the EU institutions is still limited. Lack of consideration for disability-related issues since the outset of policy initiatives has sometimes resulted in burdensome adaptions and amendments during the policy-making process or in non-inclusive policies.

5.1.4. Success factors in implementing the EU Strategy

In relation to the effectiveness of the Strategy, the Study has identified the following elements as success factors for its implementation:

-Use of different types of actions: the list of actions of the Strategy included a mix of different types of legislative and non-legislative initiatives, which have been complementary in achieving the Strategy’s objectives. Legislative initiatives are binding in nature but have long implementation periods. Non-legislative initiatives (such as awareness raising, exchange of best practices and peer reviews) although non-binding, tend to yield results within shorter periods.

-Integration with existing policy and legislative instruments: the Strategy leveraged on existing instruments to implement and mainstream disability at EU level.

oIntegration with EU funds: funding legislation for the period 2014-2020 introduced a set of pre-conditions to be met by national authorities before funding, and requiring administrative capacity for the implementation and application of the UNCRPD and of anti-discrimination law and policy in the field of EU Funds. This resulted in a strong engagement at local level for more inclusive funding, especially in the field of accessibility.

oIntegration of European Pillar of Social Rights and European Semester: the inclusion of disability in these two soft policy instruments allowed to further increase the visibility of disability policy at EU level and to issue specific disability-related recommendations in the context of the European Semester, thus creating stronger coordination between EU and Member States on disability policy.

-Exchange of good practices and awareness-raising: the needs of persons with disabilities are similar across all countries. The exchange of good practices allowed individuals, organisations and governments to rely on a wide array of experiences and knowledge at EU level. Awareness raising activities allowed to reach types of stakeholders who are not always fully aware of disability issues (such as employers and companies). At international level, the ratification by the EU of the UNCRPD and the inclusion of external action as an area of the Strategy have made international stakeholders aware that the EU wants to engage with countries across the globe to improve the situation of persons with disabilities.

-Joint effort with civil society: the implementation period of the Strategy saw an increased level of joint work and cooperation between civil society groups and EU institutions to deliver the Strategy, both through formal institutional governance arrangements and informal activities. The joint effort resulted in positive synergies especially in the following aspects: better understanding of problems and issues at stake, development of more inclusive policies and legislation, better awareness about disability issues at institutional level.

5.1.5 Gaps and challenges that hindered achievement of the objectives

The evaluation has identified the following elements as hindering factors for the implementation of the Strategy:

Design issues:

-Lack of a comprehensive set of indicators and benchmarks: the Strategy had no comprehensive indicators and benchmarks to assess the degree to which its objectives were achieved. This makes it difficult to provide an objective evaluation of its progress and to set up adequate monitoring mechanisms.

-No monitoring framework: the Strategy had no structured and operative framework for regularly monitoring or evaluating its implementation. A first assessment of the Strategy progress only took place in 2017 on the occasion of the mid-term review, when the Strategy was well under way. The implementation of a monitoring framework (with a dedicated baseline, data collection tools, a clear timeline and measurable objectives) would have allowed regular evaluation of its progress, early detection of delivery gaps and identification of possible mitigating actions. Such a framework, if implemented, would have also allowed the provision of regular and transparent information to external stakeholders about the progress of the Strategy.

-Limited focus on non-visible disabilities and children’s rights: actions, particularly in the area of accessibility, mostly focused on visible disabilities. The needs of persons with intellectual disabilities and with non-visible disabilities in general, have been taken into account to a lesser extent. These considerations also apply to children’s rights, which were not adequately included in the Strategy.

-Limited scope in relation to the UNCRPD: several stakeholders consider that the Strategy’s scope did not capture the full range of rights of persons with disabilities, and that the Strategy should be aligned with the UNCRPD. The Strategy does not cover topics such as right to life, freedom from exploitation, violence and abuse, respect for privacy, respect for home and the family, and does not address women and children with disabilities, as well as people with disabilities from a minority or with a migrant/refugee status. Some stakeholders, especially at Member State level, consider that an alignment with the UNCRPD would have made the Strategy more impactful. However, the Study also suggests that full alignment of the Strategy with the UNCRPD might have been difficult considering that many competences in the field of disability are the responsibility of Member States.

Implementation issues:

-Uneven knowledge of disability issues at EU institutional level: despite positive results in mainstreaming disability issues in policy and legislation, general knowledge about disability and about the UNCRPD across the European Commission as well as across other EU institutions is still uneven. The Study suggests that the absence of dedicated focal points in the Commission services has limited the understanding of crosscutting issues and, ultimately, the effectiveness of the Strategy.

-Limited inclusion in policy-making process: disability considerations are not applied at all stages of the policy or the legislative procedure, while they should be considered since the early stages of the process.

-Limited training: the evaluation suggests that occasional training on disability has been provided to personnel working in the EU institutions, but that the training did not result in adequate general awareness on disability and in adequate knowledge among persons directly involved in disability policy-making.

-Varying capacity of Member States and Regions to implement disability policies: due to the split of competences between the EU and the Member States, a broad range of factors influenced the general outcomes and important parts of the Strategy’s impact (and achievement of the Strategy’s objectives) depended on Member States action. Following the global financial crisis and the recession, there were widespread public spending cuts, which reduced the capacity of some Member States to implement actions related to the Strategy.

-Limited knowledge sharing at stakeholder level: there has been a significant volume of research and information on good practice generated since 2010 across the eight areas of action. However, coordination of relevant partners at EU level and between Member States in mutual learning has been carried out only to a limited extent. Better knowledge sharing would have maximised the impact of available information and would have created better awareness and stronger connections between policy-makers and experts.

5.1.6 Engagement of stakeholders in the Strategy’s implementation

The Study suggests that Member States, EU institutions and organisations representing persons with disabilities have been adequately engaged in the Strategy’s implementation via institutional mechanisms, public events, public consultations and informal meetings. These initiatives bring together a high number of stakeholders. They facilitate the exchange of information and ideas on policies and practices, and they stimulate the creation and the consolidation of disability networks.

The Study suggests that the most effective fora for engaging with stakeholders at EU level are the following:

-Disability High Level Group: meeting twice a year, it involves representatives of Member States’ institutions, the Commission, the Council of Europe and civil society on issues relating to disability and aims at facilitating the exchange of information, experiences, and good practices in the areas of disability and the implementation of UNCRPD as State Parties.

-Work Forum on the Implementation of the UN Convention on the Rights of Persons with Disabilities in the EU and the Member States: is an event organised annually since 2010. The Forum involves persons with disabilities through their representative organisations, focal points, coordination mechanisms, monitoring mechanisms of the UNCRPD from the Member States and the EU and relevant international bodies. It specifically focuses on the implementation of the UNCRPD through exchanges on concrete measures undertaken by the States Parties and dissemination of information.

-Strategic Dialogue Meetings: these meetings are organised on a monthly basis with civil society organisations, notably the ones active in the field of disability pursuing collective interests, and aim at advancing ideas in the field of disability in the areas of employment, inclusion and social affairs.

-Annual Conference for Inclusive Growth: organized once a year, it is an action-oriented platform bringing together civil society organisations and policy-makers active in the field of employment, inclusion and social affairs to discuss how to achieve truly inclusive growth. The conference systematically includes a workshop related to specific disability issues related to the main topic of the annual conference, in order to ensure that the disability angle is taken into account in the debates and conclusions.

-European Day of Persons with Disabilities: annual international conference gathering over 400 participants (persons with disabilities and their families, organizations representing persons with disabilities, experts, EU officials, politicians) from all over the EU to discuss specific themes of importance for persons with disabilities and ensure awareness-raising about disability policies at the EU and Member States levels.

In terms of wider participation and in line with the Better Regulation provisions, the launch of the evaluation of all major EU policy and legislative initiatives (including the launch of the Strategy, its mid-term review, its final evaluation and the launch of any initiative having impact on disability) includes a wide consultation process involving all relevant stakeholders. Furthermore, private citizens are given the opportunity to express their views through regular on-line public consultations published on the ‘Have your say’ section of the Commission website.

The online public consultation launched in 2019 on the occasion of the final evaluation of the strategy saw an increased participation by citizens and organizations compared to the mid-term review of 2017 (2,547 participants in 2019 versus 1,518 participants in 2017). In 2019, 1,237 respondents were persons with disabilities or caring for a person with disabilities. In 2017 their number was 563.

Consultation with all relevant stakeholders, when carried out, has proven beneficial for both the stakeholders and the Strategy by providing a constant flow of feedback, opinions and ideas about all aspects of Strategy’s implementation, with important insights for the preparation of the future disability policy.

However, in terms of stakeholder engagement, the study has also identified the following shortcomings:

-Civil society organisations have not always been involved throughout the whole lifecycle of disability-related initiatives, especially in their conception phase.

-Current fora do not always involve stakeholders representing the interests of persons with disabilities in the workforce (such as trade unions and support organisations of workers).

-Meetings of the Disability High Level Group do not provide for sufficient space for exchange of experiences, benchmarks and good practices.

More information on stakeholders’ consultation activities in the context of this evaluation is available in Annex 2.

5.2 Efficiency

5.2.1 Cost-effectiveness of the Strategy

An overall assessment about the cost-effectiveness of the Strategy is difficult to quantify in objective terms. The Strategy is a soft-policy instrument with no dedicated budget and its results, which concern the participation of persons with disabilities in society and economic life, cannot easily be measured in economic terms. Moreover, most of the actions and objectives of the Strategy are carried out by different actors.

As a general conclusion, the evaluation found that the Strategy was designed and implemented efficiently. This conclusion is based on the analysis of the costs of the Strategy and on feedback provided by relevant stakeholders in relation to its effectiveness.

The evaluation also suggests that EU funds were effective in promoting and implementing de-institutionalisation as well as in supporting accessibility and social inclusion for persons with disabilities.

As for other financial resources, grants to organisations representing persons with disabilities at EU level and funds for organising public events (such as the European Day of Persons with Disabilities, the Annual Conference on Inclusive Growth or the Access City Award) have proven highly effective in stimulating debate and in raising awareness about disability at EU level.

5.2.2 Adequateness and proportionality of resources

The Study suggests that the resources allocated to the Strategy were not fully adequate and proportionate.

Many stakeholders expressed the view that resources were not adequate for achieving the Strategy’s objectives. This opinion was more common among persons who declared having limited knowledge of the Strategy rather than among individuals having knowledge of the Strategy and among organisations representing persons with disabilities.

On a positive note, the Study suggests that the Strategy has contributed to the increase of overall resources for disability-related issues, by raising general visibility and awareness on disability.

EU funds remain the main source of funding, but the lack of clear and binding links between the Strategy and the EU funds was sometimes detrimental to the proportionate allocation of resources and their use for disability-related initiatives. In fact, many EU-funded initiatives, once implemented at national level, followed a broad-based approach aimed at solving general social problems rather than focusing on specific issues. As a result, disability-specific needs were sometimes overlooked during the implementation of those initiatives.

5.3 Relevance

5.3.1 Relevance of the Strategy in addressing the needs of persons with disabilities

The evaluation suggests that the Strategy has been relevant and continues to be relevant in addressing the rights and needs of persons with disabilities and in influencing EU law and policy processes.

The EU’s ratification of the UNCRPD means that the Convention is part of the EU legal order. All EU Member States have also ratified the Convention. The Strategy, as the main instrument to implement the UNCRPD at EU level, helps to ensure that EU policies and initiatives relevant for persons with disabilities are designed and implemented in line with the UNCRPD provisions. Member States are also required to present their strategies to implement the Convention within the areas for which they are responsible and competent.

The European Pillar of Social Rights and the European Semester are policy instruments used to coordinate economic and social policy processes across the EU. The Strategy has been instrumental in shaping and giving practical implementation to many principles of the Pillar, and specifically to principle 17, dedicated to the inclusion of persons with disabilities 100 . In relation to the European Semester, the Strategy has provided a policy framework for identifying actions and recommendations to be included in the Semester’s process and for pursuing the overall goal of the Europe 2020 strategy to create ‘smart, sustainable and inclusive growth'. As persons with disabilities experience significant barriers in education, employment, and social protection, the Strategy has been relevant to pursue the European 2020 headline targets relating to employment, education, poverty and social exclusion.

The Study suggests that the Strategy remains relevant in addressing the needs of persons with disabilities because it guides EU work in the implementation of the UNCRPD and it ensures mainstreaming of disability across all EU policies.

However, the Study also suggests that the content of the Strategy had a strong operational nature due to the emphasis on detailed and numerous implementation actions, and that the role of its policy provisions was rather limited.

5.3.2 Relevance of the Strategy for its different stakeholders

The Study suggests that the Strategy was highly relevant for institutions and stakeholders at EU level, due to its role in implementing the UNCRPD at EU level, but had limited visibility at Member State and citizen’s level.

EU level stakeholders also widely agree that the areas of the Strategy remain relevant also for the future (with a relatively lower score for external action, that might be due to the fact that external action in the disability area is a priori less visible and impactful for EU stakeholders).

Figure 4: Relevance of the current areas of action for future disability strategy

Source: ICF analysis

On the other hand, key policy-makers at Member State level seem to have prevalently some familiarity or no familiarity with the Strategy, which suggests a relative influence of the Strategy at this level.

The table hereunder provides a breakdown about the level of familiarity with the Strategy among policy-makers and organizations representing persons with disabilities in the Member States.

Table 2: Familiarity with the Strategy in the Member States

How familiar with the Strategy are the policy makers (government) in your country?
Very Familiar	Somewhat familiar	Not familiar
PT, UK	BE, BG, CY, CZ, DE, DK, EE, FR, EL, HR, LU, MT, RO, SI, SK	AT, ES, FI, HU, IE, IT, LT, LV, NL,PL, SE
How familiar with the Strategy are organisations representing persons with disabilities in your country?
Very Familiar	Somewhat familiar	Not familiar
DK, RO	AT, BE, BG, CY, CZ, DE, EE, ES, FR, EL, HR, PT, SE, SK, UK	FI, HU, IE, IT, LT, LU, LV, MT, NL, PL, SI

Source: Member State analysis performed by national experts in the context of the Study

5.4 Coherence

5.4.1 Internal coherence of the Strategy

The evaluation suggests that there was a good level of coherence among the eight thematic areas and the four implementation instruments from a policy design perspective as they incorporate key issues relevant to improve the situation of persons with disabilities in the EU.

From a policy design perspective, the Strategy combined well actions to remove barriers and empowering persons with disabilities to participate in society. It covered the fundamental issues of accessibility, participation, and equality. Removal of barriers in these areas are pre-conditions for success of specific measures related to employment, education and training, social protection and health.

In this sense, the Strategy provided a coherent, stepwise framework to address issues that prevent persons with disabilities from playing a full role in society.

The implementation instruments have proven to be a valid support for implementing specific measures included in the Strategy. For example, improved statistics helped improving targeting and assessment of the impact of particular actions, and funding supported delivery of particular actions.

However, the Study suggests that, from an operational perspective, the objectives are not always coherent with the related actions of the Strategy because some of them are formulated in ambitious terms. If taken literally, their achievement would have required actions with a much wider scope, funding and strength, in terms of capacities as well as of outreach potential, than those of the Strategy.

This consideration particularly applies to the overall aim of the Strategy (empower people with disabilities so that they can enjoy their full rights) and to the objectives in the areas of participation (achieve full participation of people with disabilities in society by enabling them to enjoy all the benefits of EU citizenship, removing administrative and attitudinal barriers to full and equal participation, providing quality community-based services, including access to personal assistance) and equality (eradicate discrimination on grounds of disability in the EU).

5.4.2 Coherence of the Strategy with other EU policies/ actions (Europe 2020, the European Pillar of Social Rights, the European Semester, the Charter of Fundamental Rights of the European Union and the EU funding provisions)

In relation to external coherence of the Strategy, it is important to distinguish between instruments that pre-date the Strategy (e.g. the Charter of Fundamental Rights) and those that were adopted after the Strategy (e.g. the European Pillar of Social Rights).

Considering that the EU ratified the UNCRPD in 2010 and that the Strategy provided a clear policy framework in the same year, EU policies and actions adopted after 2010 are generally aligned with the Strategy. However, it is important to note that the Strategy was not updated after its mid-term evaluation in 2016, therefore policy developments (e.g. Agenda 2030 for Sustainable Development) or the Concluding Observations of the CRPD Committee were not incorporated in the Strategy.

-Europe 2020: is the EU’s economic strategy for a smart, sustainable and inclusive growth covering the same period as the Strategy. In general terms, the degree of correlation between the two instruments is not strong. In fact the only mention of disability in Europe 2020 is under the flagship initiative European Platform against Poverty, where Member States are called to define and implement measures addressing the specific circumstances of groups at particular risk, including persons with disabilities. However, the disability angle, although not highlighted in Europe 2020, is particularly relevant from an inclusive growth point of view because persons with disabilities are usually at a higher risk of poverty, exclusion and unemployment. It can be concluded that, in relation to Europe 2020, the Strategy played an important complementary role by helping to focus on the achievement of Europe 2020 objectives from a disability point of view.

-The European Pillar of Social Rights: was proclaimed in 2017 to serve as a compass for delivering new and more effective social rights for citizens. Principle 17 on inclusion of persons with disabilities states that “persons with disabilities have the right to income support that ensures living in dignity, services that enable them to participate in the labour market and in society, and a work environment adapted to their needs”. The Pillar therefore has a high degree of coherence with the Strategy and it thus provided a broader context for the implementation of the Strategy actions and for mainstreaming its impact for several areas of the Strategy (social protection, accessibility, participation, employment).

-The European Semester: is the EU’s annual cycle to monitor economic and social progress, launched in 2010. During the first years of the cycle disability issues were not frequently referenced. Some recommendations, particularly those oriented towards reform of social protection and austerity measures, have actually been detrimental to reaching the objectives of the Strategy because they led to a reduction of public expenditure. However, considerations on disability have increased in recent years and were included in country reports and in the recommendations addressed to Member States. It can be concluded that in the early years provisions of the European Semester were not coherent with the overall purpose of the Strategy and have partly hindered its implementation. However, in recent years the European Semester has increasingly aligned its provisions with the aims of the Strategy, thus providing a good level of coherence.

-The Charter of Fundamental Rights of the European Union mentions disability in Article 21 on Non-discrimination and Article 26 on Integration of persons with disabilities. Article 21 prohibits discrimination on several grounds, including disability. Article 26 states that “The Union recognizes and respects the right of persons with disabilities to benefit from measures designed to ensure their independence, social and occupational integration, and participation in the life of the community”. The Strategy is coherent with the Charter, and plays a complementary role for the practical implementation of its principles.

-The EU funding provisions: the implementation period of the Strategy covers two Multi-Annual Financial Frameworks (MFF), the 2007-2013 and the 2014-2020. In the current period, there are provisions on requiring that EU Funds are used to promote accessibility, non-discrimination, and respect for human rights of persons with disabilities. The European Social Fund (ESF) supports the inclusion of persons with disabilities in the areas of employment, training, social innovation, social services through specific objectives, and particularly thematic objective 9 on social inclusion. The European Regional Development Fund (ERDF) prioritizes the promotion of social inclusion, de-institutionalization and the fight against discrimination, while considering the specific needs of persons with disabilities in the area of accessibility. The Rights, Equality and Citizenship programme promotes the rights of persons with disabilities and non-discrimination through specific funding to EU level disability organizations, research, and training and awareness-raising actions. The evaluation suggests that there is a strong level of coherence between the Strategy and EU funds.

5.4.3 Coherence of the Strategy with the UNCRPD

The European Disability Strategy 2010-2020 was adopted on 15 November 2010, several days before the EU acceded to the UNCRPD in December 2010. This means that technically the Strategy does not implement the UNCRPD. Nevertheless, the Convention guided the drafting process of the Strategy and policy-makers of the Commission were well aware of the UNCRPD when preparing the Strategy.

As result, the Strategy is fully aligned and coherent with the principles of the UNCRPD and the eight thematic areas covered in the Strategy closely link to specific provisions of the UNCRPD.

The Strategy does not cover all the areas of the Convention. Many stakeholders consider that this reduces the level of coherence between the two policy tools. But many aspects of disability policy are, to a large extent, of Member States competence and the Strategy’s scope focuses on areas where EU action proves more effective. Coverage of the full scope of the Convention would have required a broader scope of the strategy with the coverage of actions and policies for which the Member States are primarily responsible.

5.4.4 Coherence of the Strategy with the related policy measures in Member States

In general, national priority measures were coherent with EU level actions under the eight thematic areas of the Strategy. There were some differences among areas, with the highest level of coherence in the area of Accessibility and the lowest level of coherence in the area of Social protection.

Figure 5: Alignment between national priority measures and the areas of action of the Strategy

Source: ICF analysis of public consultation

Specific statement: The national priority measures of your Member State are in line with the 8 areas of action of the Strategy

In the area of Accessibility, consistency between the Strategy’s objectives and the policy measures at Member State level was reinforced through the adoption of binding legislative acts such as the Web Accessibility Directive, the European Accessibility Act, the passengers’ rights regulations and the EU funds regulations.

On the other hand, in relation to social protection, the economic policies adopted by the Member States during and after the financial crisis have generally reduced public spending, thus also reducing expenditure for social protection. This resulted in limited coherence between the objective of the Strategy in the area of social protection and the actions implemented at Member State level in this field.

The evaluation points out that the most influential policy framework for Member States’ disability policy is the UNCRPD. The overall coherence between Member States’ policies and the Strategy is due to their adherence to the principles of the UNCRPD.

The table hereunder reflects the most common responses from Member State reports across each area of action.

Table 3: Examples of Member State policies and comparison with the objectives of the Strategy

Areas of Action	In what way are policy objectives coherent with the Strategy?	In what ways do trends in Member States differ from the Strategy’s objectives?	What are the most influential policy frameworks?
Accessibility	National accessibility measures adopted based on European Directives such as the European Accessibility Act and Web Accessibility Directive.	Accessibility not fully achieved in accordance with EU legal standards; Member State reports mention lack of access to transport services, buildings, housing and lack of universal design.	UNCRPD, the European Accessibility Act and the Web Accessibility Directive. Some reports also mention national disability strategies and domestic laws on accessibility.
Participation	Programmes for transition from institutional to community-based care. As well as, national initiatives and legislation promoting participation in elections, cultural activities, public administration and employment.	Insufficient development of community-based services and persistence of institutionalisation. Equal participation has not been achieved e.g. low labour market participation, shortage of personal assistance services and limited agenda for independent living.	UNCRPD and national acts/legislation related to equal rights and participation.
Equality	National anti-discrimination and equal rights legislation covering persons with disabilities.	Persons with disabilities deprived of legal capacity under guardianship systems. Discrimination towards people with disabilities has not been eradicated.	UNCRPD, European anti-discrimination legislation (e.g. Directive 2000/78/EC) and the Strategy. A few countries mentioned national policies for equal opportunities for persons with disabilities.
Employment	National policies for increasing participation of persons with disabilities in the open labour market including reforms in national legislation supporting employment of persons with disabilities and use of European Social Fund for employment of persons with disabilities.	Failure to substantially increase the employment rate of persons with disabilities and lack of transition mechanisms from education to employment. Lack of statistics on the labour market situation for persons with disabilities.	UNCRPD, Directive 2000/78/EC, UN Committee recommendations and specific national employment policies.
Education and Training	National policies and legislation promoting inclusive education and life-long learning.	In some Member States, national policy does not adequately promote integration in the school system. Gaps in inclusive education such as low participation, lack of statistics on participation, limited teacher training and lack of support for life-long learning.	UNCRPD and specific national legislation and strategies.
Social Protection	National policies and legislation improving the benefits system (access to and adequacy of pensions, social security) for persons with disabilities and poverty reduction measures.	Insufficient levels of social protection, territorial disparities in level of protection, low coverage and lack of data/monitoring.	UNCRPD, budget cuts in Member States that have impacted services for persons with disabilities (typically those linked to austerity measures) EU legislation and national legislation and strategies.
Health	National policies and legislation for improved accessibility of healthcare for persons with disabilities.	Lack of priority or measures to ensure adequate access to health care services. As well as, lack of training of health care staff, persisting territorial differences and lack of data/monitoring.	UNCRPD and national health strategies.

Source: Member State analysis performed by national experts in the context of the Study

5.5 Added value

5.5.1 Added value of the Strategy in eliminating barriers for persons with disabilities and in implementing the UNCRPD

The evaluation suggests that the Strategy provided added value in helping to eliminate barriers for persons with disabilities and in helping to implement the UNCRPD

Stakeholders from the Disability High Level Group and from organizations representing persons with disabilities largely agree that the Strategy has delivered added value in implementing the UNCRPD.

Figure 6: Extent to which the Strategy has delivered added value in implementing the UNCRPD

Source: DG EMPL survey of Disability High Level Group (DHLG) and Non-Governmental Organisations (NGOs)

Specific statement: The Strategy has delivered added value in helping eliminate barriers for persons with disabilities and in helping implement the UNCRPD compared to what would have been achieved without it.

In fact, the ratification of the Convention by the EU and its Member States created momentum for a more ambitious and effective disability policy, and the absence of a strategic framework on disability would have left the decisions regarding implementation of the UNCRPD to the agendas of single services, with possible negative effects in terms of internal coherence.

As a result, the main added value of the Strategy was to provide coherence to the implementation of the UNCRPD at EU level.

At Member State level, the evaluation suggests that the Strategy had a more limited effect, mostly due to the limited competences of the EU in a number of policy fields relevant to disability (for instance health and education and training) and to the limited knowledge of the Strategy by national policy-makers.

An exception to this is the European Semester, a process whereby individual Member States receive recommendations on economic and social measures to be implemented. During the last years, a growing number of recommendations regarding disability has been included in the recommendations addressed to Member States. On top of the actions put in place by the Member States to implement the recommendations, an important side effect of these recommendations was the promotion of policy debate about disability and about the UNCRPD in the Member States.

Overall, it is not easy to quantify to what extent the Strategy has translated into significant impact in the elimination of barriers for persons with disabilities. The lack of disaggregated data on disability and the long-term effect of some of the Strategy’s actions make it difficult to assess progress in the situation of persons with disabilities.

However, based on the feedback received by stakeholders and on the results of the public consultation, it appears that the majority of stakeholders agrees that EU action has helped to improve the situation of persons with disabilities in the last ten years.

Figure 7: Role of EU initiatives in improving the situation of persons with disabilities in the last ten years

Source: ICF analysis of public consultation

Specific statement: Overall, EU initiatives have helped to improve the situation of persons with disabilities in the last ten years

6.Conclusions

The objective of this evaluation was to review the implementation of the European disability strategy 2010-2020 and to assess to which extent the Strategy translated into sound policies and planned changes, as well as the way it influenced implementation of the UNCRPD at EU level.

The evaluation covered the effectiveness, efficiency, relevance, coherence and EU added value of the Strategy. These criteria were examined in an external study, and built upon by the information gathered through consultation activities, including an open public consultation, targeted consultations and interviews with a range of relevant stakeholders and various other secondary sources.

Overall conclusion

The evaluation concludes that the Strategy made a significant contribution to the implementation of the UNCRPD at EU level and to the implementation of several important legislative and policy instruments in the field of disability.

However, not all the actions of the Strategy were fully implemented, and its objectives were partially achieved.

Main achievements

The Strategy had a positive impact on overall disability policy, as it contributed to a change towards a human rights-based approach in this area. The existence of a high-level EU policy document in this field significantly contributed to placing the disability agenda higher among the EU policy priorities and it opened the EU institutional space to persons with disabilities.

Being the tool for implementing the UNCRPD at EU level, the Strategy has supported the implementation of the Convention by translating its principles into practical objectives and actions.

During its 10 year implementation period, the Strategy was a driver for including disability in EU legislation and policy. The influence of the Strategy on legislative and policy instruments was particularly visible in the areas of accessibility and passengers’ rights, with the adoption of the European Accessibility Act , the Web Accessibility Directive and legislation on the Rights of Passengers with Reduced Mobility .

There was also visible impact through EU funding in the area of support to de-institutionalisation, as well as of accessibility, support for social inclusion and promotion of the rights of persons with disabilities. Key outputs included awareness raising and civil dialogue events contributing to the promotion of the rights of persons with disabilities.

The Strategy was a driver for helping in mainstreaming disability issues at EU institutional level. Examples of disability mainstreaming include development cooperation, standardisation, State Aid, copyright legislation, education programmes. Disability was also mainstreamed throughout the principles of the European Pillar of Social Rights and the disability-related Country Specific Recommendations under the European Semester, thus supporting Member States in applying the principles of the UNCRPD at national level.

The Strategy, through the establishment of the Mechanisms required by the UNCRPD, made an important contribution to ensuring the involvement of persons with disabilities and their representative organisations in the EU decision-making. The resulting exchange of feedback, opinions and ideas about disability policy contributed to a better understanding of problems and issues at stake, the development of more inclusive policies and legislation, and an increased awareness about disability issues at institutional level.

Main shortcomings

The main shortcoming of the Strategy is related to aspects of its design. The Strategy, since its conception, did not include a comprehensive set of benchmarks and indicators for measuring the extent to which its actions were carried-out and its objectives achieved. It was also missing an operative framework for regularly monitoring its implementation. The absence of those two elements, together with the limited availability of regularly collected and disaggregated data in all areas of disability, prevented the regular evaluation of the Strategy’s progress, the early detection of delivery gaps and the identification of possible mitigating actions. Such a framework, if implemented, would have also allowed the provision of regular information to all involved stakeholders and its absence is one of the reasons for the partial achievement of the Strategy’s objectives.

Despite the cross-cutting nature of disability policy and the ratification of the UNCRPD by the EU, knowledge of disability issues across the EU institutions remained uneven. Despite the European Commission’s role as focal point for implementing the UNCRPD at EU level, awareness of the UNCRPD and of the needs of persons with disabilities are not equally present across its different services; similar considerations on the need to improve knowledge and awareness around disability can be made for the other EU institutions. As a result, UNCRPD provisions and needs of persons with disabilities were not always fully taken into account in the EU policy-making processes.

Actions, particularly in the area of accessibility, mostly focused on visible disabilities. The needs of persons with intellectual disabilities, and with invisible disabilities in general, have been taken into account to a lesser extent. These considerations also apply to children’s rights, which were not adequately included in the Strategy.

Overall, the Strategy achieved partial results in implementing the foreseen actions and in reaching the set objectives. Only two thirds of the actions were fully implemented, and the majority of the objectives for the areas of the Strategy were not fully achieved. The main reasons identified in this respect are:

-The lack of a framework for regularly monitoring the Strategy’s progress prevented a regular review of implementation and the adoption of corrective actions.

-Some of the actions, although implemented, did not bring the expected results.

-Some of the objectives, especially in the areas of equality, were rather ambitious, notably because they were supported by actions that were not always sufficient for their achievement.

-Some of the EU actions were conceived to supplement and support Member States action and part of the Strategy’s impact (and achievement of the Strategy’s objectives) depended on Member States. However, following the global financial crisis and the recession there were widespread public spending cuts that reduced the capacity of some Member States to implement actions related to the Strategy.

-The financial resources available at EU level for disability-related actions were not adequate and proportionate.

-In spite of the different benefits of EU funding, the lack of clear and binding links between the Strategy and the EU funds was sometimes detrimental to the proportionate allocation of resources and their use for disability-related initiatives. In fact, initiatives having a clear purpose (such as, for example, funding de-institutionalisation) were more effective in addressing disability issues than those focusing on general social issues.

Lessons learned

Realistic objectives supported by adequate actions: the Strategy included certain ambitious objectives which were not always supported by equally ambitious actions. This sometimes resulted in implementation gaps between the objectives of the Strategy and the achieved results. Objectives and actions should thus be better aligned.

Measurable policy indicators: it is necessary to ensure a comprehensive set of quantitative indicators, of a clear timeline and of a monitoring mechanism for measuring progress of the Strategy in order to create a momentum for addressing delays and weaknesses in the course of the implementation of the Strategy.

Statistics and data collection: a comprehensive framework of statistic indicators covering all the key priority areas of the Strategy is needed to identify gaps, to measure progress in the situation of persons with disabilities in the EU and to set clear targets for the Strategy. In general terms, EU-wide disaggregated data on persons with disabilities would be an important tool for monitoring and evaluating European policies aimed at improving the situation of persons with disabilities.

Disability knowledge: knowledge by policymakers at EU institutional level about disability issues and about the UNCRPD has to be equally present across services. Unequal knowledge proved to be an obstacle for inclusive and efficient policymaking.

Involvement of persons with disabilities: consulting and involving persons with disabilities (and notably organisations representing persons with disabilities) from the start of any legislative and policy initiative relating to persons with disabilities is a key element for delivering effective and inclusive policy making.

Progress at national level: a comprehensive overview and monitoring of progress regarding the situation of persons with disabilities requires an evaluation of EU-level action as well as of national-level action. Limitation of the evaluation of the current strategy to EU-level action had consequences for the implementation results of the Strategy. This element will have to be taken into account in future policy decisions.

Annex 1: Procedural information

1.Lead DG, Decide Planning/CWP references

DG Employment, Social Affairs and Inclusion (DG EMPL) is the lead DG for the evaluation of the Council Recommendation on the European Disability Strategy 2010-2020. The initiative was published in Decide on 03 September 2018 with the reference PLAN/2018/3426 101 .

2.Organisation and timing

The evaluation started on 28 September 2018 with the publication of the Evaluation Roadmap 102 and has been carried out with the support of the Inter Service Group chaired by DG EMPL to which the following DGs were invited: AGRI, BUDG, CNECT, COMM, COMP, DEVCO, DGT, EAC, ECFIN, ECHO, EMPL, ENV, ESTAT, FISMA, GROW, HOME, HR, IAS, JRC, JUST, MOVE, NEAR, OP, REGIO, RTD, SANTE, SG, SJ, TAXUD and TRADE.

The group met or was consulted through written consultation five times.

Here below are the key steps of the evaluation:

-13/09/2018 - evaluation roadmap and terms of reference for the external study

-17/06/2019 - inception meeting for the external study

-17/10/2019 - draft interim report of the external study

-18/11/2019 - interim report of the external study

-19/03/2020 - written consultation on the draft final report of the external study

-05/08/2020 - draft SWD

-09/09/2020 - written consultation on the draft final report of the external study and on the draft SWD

-15/10/2020 – inter-service consultation on the draft SWD

-20/11/2020 – publication of the SWD

3.Exceptions to the better regulation guidelines

The consultations on the draft final report of the external study and on the draft Staff Working Document were carried out by written procedure over email due to coronavirus lockdown.

4.Evidence, sources and quality

Expertise internal and external to the European Commission was used to ensure good quality of the evaluation and related Staff Working Document. The main sources of the evaluation include:

·External study carried out by ICF consortium. The study has been conducted in line with the Request of Services and the agreed inception report. It includes all agreed components and is based on relevant qualitative and quantitative data although less information and evidence on effectiveness was included than expected. The analysis and conclusions are sound while methodology and limitations are clearly outlined. Conclusions and recommendations are relevant.

·An internal questionnaire by DG EMPL on the European disability strategy 2010-2020 addressing NGOs at EU level

·An internal questionnaire by DG EMPL on the European disability strategy 2010-2020 addressing representatives of Member States at EU level

·Studies/toolkits produced by the Academic Network of European Disability Experts (ANED) 103

·Reports from the European Days of Persons with Disabilities 2019 104

·Opinion from the European Economic and Social Committee on Shaping the EU agenda for disability rights 2020-2030 105

·Motion for a resolution of the European Parliament on the European Disability Strategy post‑2020 106

·European Union Statistics on Income and living conditions – EU SILC 107

Annex 2: Stakeholder consultation

This annex presents an overview of the consultation work conducted for the evaluation of the European Disability Strategy 2010-2020. Most of the consultation activities were carried out by an external contractor 108 on behalf of DG Employment, Social Affairs and Inclusion (DG EMPL).

Prior to the evaluation, an Evaluation Roadmap 109 was published for consultation between 29 August and 26 October 2018. The 13 respondents to the consultation included 10 non-governmental organisations, 2 business associations, and 1 private citizen.

The feedback received on the Roadmap suggested that:

-EU Disability Strategy should recognise there are different forms of disability which impact on citizens in various ways, requiring varied and personalised forms of intervention and support;

-Persons with disabilities should be involved in the evaluation and they should be consulted in the drafting of the online consultation;

-Targeted consultation should offer ensure full accessibility in terms of languages as well as of disability-related needs, in order to reach a larger number of respondents;

-In terms of proposals for a new European Disability Strategy, the role of the social economy in integrating persons with disabilities into society and the economy should be promoted.

These aspects were considered in the design and delivery of the evaluation. The consultation activities undertaken during the evaluation were the following:

-A public consultation published on the European Commission’s ‘Have your say’ portal and open to all EU citizens and organisations;

-Semi-structured interviews with EU stakeholders, selected by the contractor in cooperation with Commission services. This included the following types of stakeholders: European Institutions, UNCRPD Coordination Mechanisms and Monitoring entities, Organisations representing persons with disabilities, Non-governmental organisations, Social partners, Academia and education, Service providers/ private business representatives, and Equality and human rights bodies;

-Semi-structured interviews with Member State stakeholders and focus groups with Member State stakeholders.

1.Target groups and consultation undertaken for the evaluation

Three different target groups were established for the evaluation:

1.The general public

2.EU level stakeholders

3.Member State level stakeholders

Each method targeted specific stakeholders to collect particular forms of data. An overview of the target groups, consultation methods, and those consulted is provided below in Figure 1.

Figure 1: Consultation methods and their target groups

Source: ICF analysis

2.Public consultation

The public consultation was launched on 31 July 2019 and was closed on 13 November 2019. It was hosted on the European Commission website. The public consultation collected views from a wide range of respondents (including stakeholder organisations and interested citizens living in the EU) on their perceptions of the European Disability Strategy, its implementation to date, and their opinions on important future priorities in disability policy.

The questionnaire was developed based on discussions with DG EMPL and pertinent Commission services to ensure that the public consultation would meet the requirements of the evaluation and provide the European Commission with a useful evidence base on which to base further actions in the disability policy area.

To disseminate the public consultation, the contractor worked with Member State experts to promote the public consultation to key stakeholders and their networks at both EU and Member State levels. In addition, the contractor worked with the European Disability Forum (EDF), an umbrella organisation of persons with disabilities, to promote the public consultation. This was done through the creation of social media posts in different EU languages and by sending these out through the EDF social media channels.

Figure 2: Public consultation – respondents per country of origin

Source: ICF analysis of public consultation

A total of 2,547 responses were submitted from all EU Member States. Respondents included Academic/research institutions, Business and Consumer organisations, Environmental organisations, Non-governmental organisations (NGOs), Public authorities, Trade unions and other forms of stakeholders. A total of 573 (22.5%) responses were from organisations, whereas 1974 (77.5%) responses came from individuals. Two versions of the questionnaire, a standard version and an easy-to-read version, were published, each accessible in all 24 official EU languages.

In addition to the responses received to the public consultation questionnaire, a total of 107 position papers were received. Among the position papers, 37 were in English and 70 were in other EU languages. A process for examining each of these documents was followed, with the documents analysed by a native speaker to determine their relevance to each evaluation criterion. Of the one hundred and seven documents were submitted, forty-nine contained relevant information for the evaluation. Key point summaries were prepared for useful documents that could be incorporated into the evaluation’s evidence base.

Results from the public consultation

Effectiveness

The public consultation provided some useful data with which to understand overall progress for persons with disabilities and the contribution of the Strategy:

-Overall, 56% of respondents agreed or somewhat agreed that the situation for persons with disabilities improved over the past 10 years in their Member State, with 52% of respondents agreeing or somewhat agreeing that the contribution of the Strategy helped in this improvement.

-Among persons with a disability 52% agreed or somewhat agreed that the situation for persons with disabilities improved in the past 10 years in their Member State, 48% agreed or somewhat agreed that the Strategy contributed to improving the situation.

-74% of organisations and 51% of individuals agreed or somewhat agreed that the situation of persons with disabilities improved over the past 10 years in their MS. 63% of organisations and 47% of individuals somewhat agreed or agreed that the Strategy has helped to improve the situation for persons with disabilities in their MS.

-Of those stating they had knowledge of the Strategy 33% agreed or somewhat agreed that the situation for persons with disabilities had improved over the last 10 years in their Member State, 52% of such respondents agreed or somewhat agreed that the Strategy helped to improve the situation for persons with disabilities.

The table overleaf summarises the effectiveness results from the public consultation for different areas of the Strategy. Key points which emerge from this analysis:

-The area of accessibility (66%), followed by participation (55%), education and training (54%), and equality (51%) are the areas where most respondents identify that greatest progress has been made in improving the lives of persons with disabilities.

-The areas of employment (40%) and health (41%) are the areas of action where lowest levels of progress for persons with disabilities were observed by survey respondents. In the area of health 46% of persons with disabilities disagreed there had been improvement, with 5% stating the same in the area of Employment.

-For each area of action, a greater number of respondents with no disability agree or somewhat agree that there has been progress for persons with disabilities than those with a disability.

Table 1: Public consultation - effectiveness results for the different areas of the Strategy

Area	Opinion	Respondents with knowledge of the Strategy	Respondents with disabilities	Respondents without disabilities	Organisations	Individuals
Accessibility	Agree / Somewhat agree	66%	63%	74%	50%	48%
	Somewhat disagree / disagree	22%	25%	12%	37%	40%
Participation	Agree / Somewhat agree	55%	51%	69%	46%	40%
	Somewhat disagree / disagree	28%	19%	15%	38%	43%
Equality	Agree / Somewhat agree	51%	47%	65%	45%	42%
	Somewhat disagree / disagree	33%	37%	16%	33%	44%
Employment	Agree / Somewhat agree	40%	37%	52%	39%	38%
	Somewhat disagree / disagree	41%	45%	26%	39%	44%
Education and Training	Agree / Somewhat agree	54%	53%	60%	43%	46%
	Somewhat disagree / disagree	30%	33%	21%	41%	39%
Social Protection	Agree / Somewhat agree	48%	44%	64%	47%	39%
	Somewhat disagree / disagree	35%	40%	14%	33%	45%
Health	Agree / Somewhat agree	41%	38%	53%	37%	32%
	Somewhat disagree / disagree	41%	46%	22%	41%	52%

Source: ICF analysis of public consultation

Efficiency

Views expressed through the public consultation relating to extent to which the resources provided for the Strategy were adequate were somewhat mixed:

-An average of 22% of respondents with knowledge of the Strategy agreed that funding for each area of action was adequate (39% disagreed or somewhat disagreed). The proportion of respondents who “agreed” or “somewhat agreed” that the resources were adequate varied between 15% (for the area of external action) and 26% (for the area of accessibility).

-The proportion of respondents that did not agree that funding for the Strategy was adequate ranged from 28% (for the area of action “external action”) and 44% (for the area of employment).

-Respondents with disabilities or those caring for someone with a disability were more likely to disagree that the Strategy was adequately resourced than those with no disability. Additionally, a higher share of organisations than individuals agreed that the Strategy was adequately resourced.

Coherence

In relation to the coherence of the Strategy with measures established in Member States, data analysis of public consultation responses shows that:

-Around 40% of the respondents agree, or somewhat agree that the national priority measures of their country are in line with the 8 areas of action of the EDS. This view was most frequently expressed by organisations (50%) and least frequently by individuals (30%) and those with disabilities (35%).

-When looking at the different areas of action of the Strategy there is significant difference in the views of survey respondents between the areas of the Strategy:

-In the specific area of accessibility, a greater share of respondents (51%) than for other areas of the Strategy agreed/somewhat agreed that there was alignment between national policy and the Strategy.

-Of those respondents that disagreed/somewhat disagreed that national priorities were aligned with the Strategy (36% on average), the most common areas of the Strategy that were identified as lacking coherence with national policy were: employment (43%) and social protection (40%).

Relevance

Analysis of the responses from the public consultation found that the thematic areas of the Strategy are still considered relevant for the future. Around 66% of the respondents of the public consultation found 7 of the 8 areas of the Strategy still relevant for the future, whereas 55% considered external action also relevant.

Added value

From the public consultation, the following findings emerge from this analysis:

-At Member State level, 52% of respondents agreed or somewhat agreed that the Strategy had helped to improve the situation for persons with disabilities, compared to 53% who stated the same for the effect of the Strategy at EU level.

-Relatively few respondents stated that the Strategy had not helped to improve the situation of persons with disabilities (27% stated they somewhat disagreed or disagreed it had at Member State level, compared to 11% answering the same in relation to throughout the EU).

-Interestingly, there were a high level of respondents stating that they do not know whether the Strategy had helped to improve the situation for persons with disabilities at EU level (20%) and worldwide (38%).

Important topics

Respondents were also asked to express their views about the most important topics in the area of disability. Accessibility, ageing, independent living, children with disabilities and employment were considered the most important.

Figure 3: Public consultation – important topics identified by respondents

Source: ICF analysis of public consultation

EU stakeholder interviews

Consultations with EU stakeholders took place in the form of interviews and occurred face-to-face and via telephone to accommodate availability and preference of interviewees. Interviews were conducted from September 2019 to December 2019 and lasted between 45 to 90 minutes.

These interviews with EU stakeholders helped further develop findings of the desk research and allowed stakeholders to provide evidence to answer the research questions.

The semi-structured interviews used a topic guide covering key issues to inform the evaluation framework. Individual questions were formulated to assess the relevance, effectiveness, efficiency, coherence and added value of the European Disability Strategy, to understand the current situation of people with disabilities in the EU and to gather opinions on the future direction of EU disability strategy. Interview questions were tailored according to the type of stakeholder consulted.

Twenty semi-structured interviews were conducted with EU-level stakeholders. An initial list of relevant stakeholders was identified through desk research and refined based on suggestions from the Inter Service Steering Group established by the European Commission to oversee the evaluation.

The semi-structured interviews included 17 organisations, 10 of which are directly involved in addressing the needs of persons with disabilities in general or aiming to support the needs of persons with specific types of disabilities. In general, one individual was interviewed per organisation. However, two individuals were interviewed from the European Disability Forum and three individuals from the European Union Agency for Fundamental Rights (FRA).

The table below outlines the interviews undertaken with EU stakeholders

Table 2: EU-level stakeholders interviewed

Stakeholder types	Stakeholder organisation
European Commission Institutions	European Ombudsman
UNCRPD Coordination Mechanisms and Monitoring entities	European Agency for Special Needs and Inclusive Education
Disabled Persons’ Organisations (DPOs)	Autism Europe Inclusion Europe (AISBL) European Union of the Deaf Mental Health Europe Disabled Peoples' International Europe European Platform for Rehabilitation
Non-governmental organisation (NGO)	European Disability Forum
Social partners	European Federation of Public Service Unions (EPSU) Federation of European Social Employers (FESE)
Academia and education	The Academic Network of European Disability Experts (ANED)
Service providers/ private business representatives	European Association of Service providers for Persons with Disabilities (EASPD) Community of European Railway and Infrastructure Companies Social Services Europe
Equality and human rights bodies	European Network of Equality Bodies (EQUINET) European Union Agency for Fundamental Rights (FRA)

Source: ICF analysis

3.Member State stakeholder interviews and focus groups

A network of disability experts from each Member State was established by the team of the external contractor to conduct national level consultation, including desk research, stakeholder interviews and focus groups. Focus groups were generally organised through national councils for persons with disabilities (or similar organisations in Member States where such organisations do not currently exist). Member State experts also conducted semi-structured interviews face to face and via telephone. Both interviews and focus groups were conducted between September 2019 and December 2019, with national representatives and key stakeholders for persons with disabilities, to supplement evidence gathered through desk review.

The stakeholder consultations at Member State level aimed to build on the information gathered through desk review. More specifically, to gather further information and opinions on the progress of implementation in specific areas of action, particularly focusing on national level actions motivated by the Strategy.

Outputs from these activities (i.e. desk review, semi-structured interviews and focus groups) were recorded by Member State experts in a standardised country report template. The interviews and focus groups adhered to a topic guide focused on assessing the extent of change in Member State in the issues covered by the Strategy since 2010, particularly relating to Accessibility, Participation, Equality, Employment, Education and training, Social protection and Health. In addition, the topic guide included questions to understand drivers of change in national disability policy and changes attributed to the European Disability Strategy.

A total of 162 interviews and 18 focus groups were conducted with Member State stakeholders.

4.Issues encountered and actions taken during the consultations

Throughout the course of the evaluation some challenges emerged in relation to the consultation methods. This section describes the issues and the actions taken by the team of the external contractor for each of the consultation activities.

Public consultation

-During the public consultation, there were particular access issues that inhibited the ability of respondents to answer questions. These included language defaulting to EN after some filter questions. Some stakeholders reported that the requirement to register prior to completing the public consultation deterred them from taking part to the consultation. In addition, stakeholders reported technical issues such as being unable to fill-out certain required fields, which deterred them from submitting their response. In order to mitigate the issue, the mandatory on-line authentication was de-activated.

-Low response rate in some Member States. This issue was mitigated by contacting the relevant Member State experts and urging them to encourage stakeholders to complete the public consultation and publicise it through their networks.

EU stakeholder interviews

Some stakeholders mentioned they had taken part in the public consultation and felt additional involvement through interviews was redundant. Moreover, stakeholders cited that they have been extensively involved in the previous evaluation and felt further involvement would provide no added value. These issues were flagged to DG EMPL immediately and a clear communication channel (via email, telephone and face-to-face meetings) was maintained to ensure timely response, to effectively identify and approach other relevant EU-level interviewees.

Member State stakeholder interviews and focus groups

Member State experts pointed to several issues that hindered their research activities. The timing of the evaluation was cited as an issue as Member State experts found it difficult to secure interviews or focus groups with stakeholders during the summer holiday period. In addition, some national experts mentioned that stakeholders were initially hesitant to take part assuming that their identity would be disclosed. This issue was easily mitigated by assuring both Member State experts and relevant stakeholders that their identity will be kept anonymous. To fit in which the schedule of interviewees the interviews were kept as succinct as possible, with interviewees also permitted to provide written information if that was there preferred method. In addition, some stakeholders refused to participate due to issues related to accessibility (e.g. not wanting to travel long distance or not having access to accessible technology to participate in the interview or focus group). The Member State experts who raised such issues were provided additional resources (i.e. cost of renting accessible venues and travel costs) to hold focus groups in accessible venues and hold interviews face to face in locations easily accessible for interviewees, though this was only required in three cases.

5.Lessons for future consultations

-In order to identify and prevent accessibility difficulties and technical issues, extensive piloting of the public consultation questions should be conducted involving Disabled Persons Organisations and other relevant stakeholders. This could be done through, for example, a soft launch where a small sample of stakeholders receives an invitation to complete the public consultation before it is made openly available.

-To ensure the public consultation reaches as wide as possible an audience and to obtain a good response rate, efforts can be taken to extensively promote the public consultation prior to publication. This consultation was promoted after through established social media platforms of stakeholder organisations that were familiar to persons with disabilities such as Facebook, LinkedIn, and Twitter

Annex 3: Intervention logic

Annex 4: Methods and analytical models

This annex describes the methodology used to undertake the evaluation. It firstly outlines the way in which an inclusive research methodology was applied in the evaluation and how analysis was conducted. The section then provides a brief description of the approach taken for each of the four main research tasks included in the evaluation, identifying specific issues and the way in which they were mitigated.

The research adopted an inclusive methodology

The evaluation required a range of consultations with different types of stakeholders operating at both EU and Member State level. This involved EU level stakeholder interviews, Member State level consultation (interviews and focus groups), and e-surveys such as a public consultation (the consultation was published on the European Commission’s ‘Have your say’ portal). On-line surveys were also undertaken among Non-governmental organisations (ONGs) and Member State representatives on the Disability High Level Group (DHLG).

When undertaking these consultations it was important to ensure that a wide range of persons with disabilities and stakeholder groups had the opportunity to provide input into the evaluation. An inclusive research methodology for enabling consultees to participate through forms of consultation that were appropriate for them was delivered in the following ways:

-When arranging and conducting EU level and Member State level interviews, plain language was used to describing the purpose and requirements of consultees in all forms of consultation.

-For interviews, consultees were offered the opportunity to provide input according to the most suitable method (e.g. in written format, via skype call, telephone, teleconference, or face-to-face interview).

-Where focus groups were conducted in Member States, they involved organisations representing persons with disabilities to discuss the Strategy and future EU disability strategy. Focus group attendees were provided with information prior to focus groups. This information outlined the purpose of the study and the key issues to be discussed during the focus group.

-Prior to focus groups, an assessment of participant requirements was undertaken by Member State experts to understand accommodation and participant requirements. This involved assessing the need for, and organising sign language and/or hearing loop facilities. Where appropriate, consultees in Member States were also offered the opportunity to participate in the evaluation through providing written answers or comments on the evidence assembled by Member State experts.

-The public consultation included an easy-to-read version of the questions, having a limited number of questions expressed in simple terms to enable as many persons as possible to answer the questions.

An evaluation framework was used to guide research and analysis

An evaluation framework was established during the inception stage of the evaluation. The framework used the individual evaluation questions to develop specific judgement criteria used to answer the research questions, it then identified relevant evidence and indicators (qualitative and quantitative) that would be required to make these judgements. The framework then identified specific research tasks and methods that would be used to generate the required evidence.

For each of the research questions a similar process of triangulation was followed in order to combine the results from different sources and analytical techniques. The first step in the process of analysis was to analyse the relevant information collected from each research tasks/source (document review, public consultation, member state research, EU level interviews).

-Individual sources of information were then analysed and organised by evaluation question so that they could be compared, and analysis of emerging messages and trends was undertaken.

-At the same time an assessment of the clarity of each evidence base was undertaken, though mitigating measures to address this were taken during the fieldwork stage, for example:

-encouraging more respondents to the public consultation,

-approaching more interviewees in the MS research and EU level interviews when needed,

-undertaking further document review

-Comparison of trends and patterns to understand the extent to which data sources produce similar findings has been undertaken to produce a set of emerging messages from the data analysis. Additionally, the extent to which diverging messages emerged between sources and types of stakeholders has been examined.

-The final step was to develop answers to research questions and to produce conclusions and recommendations linked to the evidence base. The answers to research questions and the conclusions and recommendations has been discussed with disability experts included in the research team and the ISG.

This process was important for this evaluation due to the range of sources of information needed to be included in the analysis (for example, views of persons with disabilities; documents such as reports, analysis, and legal instruments; consultations with stakeholders at EU and Member State level).

There were a number of issues during the research, which required mitigating action to ensure that the evidence was as reliable as possible and the findings transparent and robust. These are outlined below for different research tasks. They included, difficulty in engaging with key stakeholders; analytical challenges related to the design of the Strategy, its objectives and actions; limited information upon which to base assessment of the Strategy’s achievements and influence; and, different levels of specificity in the available evidence.

Specific research tasks, issues encountered and mitigating measures

Task 1: Mapping the progress of objectives and key actions of the European Disability Strategy

The aim of this task was to assess the progress in delivering EU level actions and to identify the extent to which the Strategy supported and supplemented Member State actions. The research has been undertaken through both desk research and targeted consultations. At EU level this involved extensive desk review and interviews with key stakeholders and consultation with the Inter-Service Steering Group to ensure that the evidence gathered during the mapping exercise was accurate. At Member State level, detailed desk research, interviews, and focus groups were undertaken.

The results of the mapping are contained in Annex 5. The results of the mapping inform the answers to various evaluation questions and are the used as evidence to answer research questions.

Issues encountered and mitigating measures

The following issues emerged, with mitigation measures put in place during the research process:

-It was not always clear from desk review and consultations the degree to which progress had been made for some of the actions, because many of the actions were not specific and measurable. This was addressed through consultation with ISG members and further targeted consultations with EU level stakeholders assisted in gaining clarification on factual matters related to implementation. It was not, however, possible to gain comprehensive information on the impact of EU level actions. In part, this is because impact may depend on timeliness and comprehensiveness of Member State level implementation.

-Dependencies between different actions of the Strategy are not always explicit, and it is not possible to provide robust evidence of the long-term impact of delivery or non-delivery of particular actions on other actions.

-In general, there is a lack of data to understand the impact of the actions and clear attribution of those impacts to the Strategy. The reasons for this are: some actions began prior to the publication of the Strategy; many actions are also influenced by other policy drivers; impact depends on implementation at Member State level and/or impact is likely to be measurable over the longer term. These issues could not be fully mitigated; therefore, the evaluation uses the assembled evidence base to provide an overall assessment of the Strategy.

Task 2: public consultation

The public consultation collected feedback from a wide range of respondents (including stakeholder organisations and interested citizens) on their perceptions of the Strategy, its implementation to date, and their opinions on important future priorities in disability policy. It ran from 31 July to 13 November 2019. The overall number of responses was 2,547; this is significantly higher than the previous public consultation undertaken for the mid-term evaluation of the EU Disability Strategy, which received 1,518 responses.

The external contractor worked with DG EMPL to develop the questions and question formats for the public consultation so that it would align with the requirements of the evaluation. An important priority for DG EMPL was to publish the public consultation before the summer so that the consultation could be completed and analysed prior to the European Day of Persons with Disabilities at the end of 28-29 November 2019.

The external contractor worked with the European Disability Forum (EDF) to promote the public consultation through their social media. To make this as engaging as possible, the external contractor created social media posts in different EU languages, with the purpose of encouraging as many responses as possible from members of the public. The European Commission also encouraged non-governmental organisations to publicise the public consultation.

A process for examining each of these documents was followed, with the documents analysed to determine their relevance to each evaluation criterion. A total number of one hundred and seven documents were submitted, forty-nine of which contained relevant information for the evaluation.

Issues encountered and mitigating measures

There were some initial issues related to respondents accessing the public onsultation that were encountered immediately following its launch. These included:

-The language defaulting to English after the initial language filter questions and some respondents not being able to enter data into some of the fields.

-A more significant issue reported by respondents was that they could not respond to the public consultation without an EU login. This requirement was removed by the European Commission. This requirement is a mandatory feature of public consultations undertaken by the European Commission and this exception required approval by the Chief Operating Officer of the Commission.

-The public consultation was launched during the summer and was initially attended by low number of respondents. As described above, the external contractor worked with Member State experts and with the European Disability Forum (EDF) to promote the public consultation as widely as possible.

Task 3: Targeted consultations

DG EMPL undertook targeted consultations held with non-governmental organisations (strategic dialogue) and MS representatives (Disability High Level Group) prior to the external contractor. These consultations were conducted using the EU survey tool, analysis of findings from these consultations are included in the evidence base used to answer the study questions.

The external contractor undertook targeted consultations at both EU and Member State level. Additionally, focus groups were arranged where possible through National Councils for Persons with Disabilities or similar organisations. At EU level, consultations were undertaken through semi-structured interviews.

The purpose of these consultations was to deepen the evidence base assembled through the desk research. The consultations provided an opportunity to also gather stakeholder opinions regarding the overall impact of the Strategy and their views on what future disability strategy should prioritise.

2,547 participants took part in the public consultation and 182 interviews were undertaken for the study (20 at EU level and 162 and Member State level). Several types of organisations were interviewed at Member State level. A full list of interviewees and focus group participants is included in the Synopsis Report.

Issues encountered and mitigating measures

Consultations were undertaken through Member State experts who provided the external contractor with bi-weekly progress reports. This enabled any issues encountered during the consultation to be understood and dealt with in a timely manner. Common issues encountered during the Member State research were:

-a low level of knowledge of the Strategy and its content;

-lack of time to participate in interviews (initially interviewees were approached over the summer period); and

-ensuring interviews (and subsequent Member State reports) remained on topic and did not focus on highly context specific issues that were beyond the scope of the evaluation.

Issues encountered during the consultations at EU level were:

-Interviewees stating that they had a lack of time to conduct interviews and stating that they felt able to contribute to evaluation of the Strategy because they had communicated their views to the Commission.

-Some potential interviewees were reluctant to take part in interviews when they felt they did not have the knowledge to discuss each of the Strategy’s areas of action.

-A belief among some stakeholders that it was not appropriate for them to participate in the evaluation (for example, where they were engaged in design and delivery of the Strategy or in separate evaluation activity).

To mitigate this, interviews were focused only on areas that interviewees were comfortable expressing an opinion and/or alternate interviewees were agreed with DG EMPL.

Task 4: Analysis by evaluation criterion

This task consisted of analysis of the available evidence generated through the research undertaken in order to answer research questions for each evaluation criterion. This followed the following process:

-Evidence was organised by research activity (document review, public consultation, stakeholder interviews, Member State research). Evidence was further broken down by evaluation criterion (effectiveness, efficiency, coherence, relevance and EU added value).

-To develop and test the emerging findings from the study, the external contractor presented selected analysis along with emerging conclusions at the European Day of Persons with Disabilities in Brussels in November 2019.

-Analysis and synthesis of the evidence for each individual research question was undertaken to determine the answers to each. Answers to study questions were then tested through consultation with experts and presented in this report.

-A summary of strengths, weaknesses, opportunities, and threats (SWOT) analysis was produced for the study.

Issues encountered and mitigating measures

A number of factors made evaluating the Strategy actions a challenging exercise. This is partly due to the nature of the objectives and actions included in the Strategy, for example:

-There are no clear performance measures for the objectives established for the overall performance of the Strategy of for its areas of action.

-Typically, the EU level actions included in the Strategy do not include explicit time-bound targets for their delivery

-Observing the effects of many actions is likely to take many years as they rely on removal of barriers/increased access, which may take a longer period for corresponding up-take to produce the desired results. For example, establishing legal instruments requires subsequent transposition and effective implementation at Member State level, there may be a similar time-lag for established tools and codes of conduct.

Annex 5: implementation status of the actions of the Strategy

This annex includes an overview of the extent to which actions have been implemented during the Strategy. The information gathered during the evaluation has been used to assess the extent to which actions of the Strategy were implemented based on some judgment criteria reported in the table hereunder.

Each action has been associated with one of the following three categories:

-F = Fully implemented: are actions adopted without major delays and in accordance with the objectives set out in their inception. This category also includes actions that were less ambitious than their original objectives suggested, but were adopted in a timely manner and actions that were not adopted in a timely manner (had some delays), but were matching or surpassing their original objectives.

-P = Partially implemented: actions not adopted timely, did not match their original objectives and were not implemented as they were envisaged. This category also includes recurrent actions that were implemented only once.

-N = Not implemented: actions that have not been adopted by the end of the evaluation.

Table 3: Judgement criteria about the implementation of the actions

Judgement of level of implementation		Judgement of level of implementation		Time considerations:		Adoption considerations		Status of actions Put in practice
Fully Implemented:	Evidence that the actions were adopted in a timely manner		Evidence that the actions were adopted in a timely manner		Actions were as envisaged / a little less ambitious than envisaged		Actions put in practice / recurrent action in place as planned
			Evidence of some delays		Adopted as envisaged or more ambitious
Partially implemented:				Evidence of some delays		Actions less ambitious than envisaged		Not implemented as envisaged / recurrent actions not operating as planned
Not implemented:				No adoption		Not applicable		Not applicable

Source: ICF analysis

EU level actions associated with Accessibility

The accessibility area of action aimed to make goods and services accessible to people with disabilities and promote the market of assistive devices. The implementation status of EU level actions is outlined in the table below.

Table 4: implementation of actions in the area of accessibility

Actions linked to the adoption of the European Accessibility Act
A cost benefit analysis and data collection study in preparation for the possible development of a European Accessibility Act	F
Consultation with Member States and other stakeholders on a possible ‘European Accessibility Act’	F
Prepare a European Accessibility Act setting out a general accessibility framework in relation to goods and services including public services	F
Explore the possibility to complete the legal framework addressing rights of persons with reduced mobility by covering all relevant modes of transport
Finalise negotiations in Council on proposal for Regulations on the Rights of persons with reduced mobility travelling by bus and coach , sea and internal waterways	F
Report on the application on the Regulations on the Rights of persons with reduced mobility travelling by air	F
Address problems related to compensation claims concerning damages of mobility equipment when travelling	P
Address accessibility of transport infrastructures through accessibility Mandate 420	N
Address accessibility to the built environment
Implementation of the Urban Mobility Action Plan	F
Promote accessibility of the built environment as part of the sustainable construction area of the "lead market initiative"	N
Implement Standardisation Mandate 420 on accessibility to the built environment	N
Exploit the opportunities provided by the Galileo system supporting pilot projects related to disabilities	F
Make full use of all existing legal instruments and address disability matters in their revisions following the UN Convention in the area of Information Society in line with the Digital Agenda for Europe
Implement the disability related provisions of the electronic Communication Directives and the Audio-visual Media Service Directive, considering their recent revisions	P
Systematically evaluate accessibility in revisions of legislation undertaken under the Digital Agenda, following the CRPD	P
Promote dialogue between users and industry leading to voluntary agreements in the areas of: copyright & digital television	F
Based on a review of options, make proposals by 2011 that will make sure that public sector websites (and websites providing basic services to the public) are fully accessible by 2015.	F
Develop accessibility standards to ICT for their use in public procurement in the context of Mandate 376	F
Support research on new technologies addressing assistive technology and accessible mainstream solutions
Mainstream accessibility following Design for all in relevant mainstream standards through a standardisation Mandate	F
Support the inclusion of accessibility and design for all in general curricula for architects and engineers to prevent barriers when exercising their professions	N
Study the Assistive technology market with a view to improve its functioning	F
Raise awareness on accessibility through the establishment of an European award of accessible cities	F
Exploit international dialogues in particular ongoing EU-US dialogue on accessibility standards to bring coherence to the markets and build economies of scale	P
Follow-up to a planned Green paper on modernisation of EU public procurement rules and explore the possibility of fostering the use of public procurement to improve accessibility by developing obligations for public authorities using accessibility standards	F
Explore possible measures to improve the accessibility of goods and services in Europe using internal market instruments	P
Consider relevant disability issues, where possible, within the context of the follow-up to the Commission Report "Towards more efficient and fairer retail services in the Internal Market for 2010" and within the context of the e-commerce activities	F
Improve application of Article 16 of the Structural funds general regulation to progress on accessibility	F
Take into consideration, in projects run by the EU Joint Research Centre, technical aspects related to usability by all, including people with disabilities	P
Support independent living of persons with disabilities and older persons under the innovation partnership "Active and healthy ageing" of the Innovation Union flagship initiative	F

Source: ICF analysis

EU level actions associated with Participation

The participation area of action aimed to: ensure that people with disabilities enjoy all benefits of EU citizenship, remove barriers to equal participation in public life and leisure activities, and promote the provision of quality community-based services. The implementation status of EU level actions is outlined in the table below.

Table 5: implementation of actions in the area of participation

Address the obstacles that persons with disabilities face in exercising their rights as individuals, consumers, students and professionals, and political actors
Study the implications of a mutual recognition of disability cards and related entitlements	F
Support Member States' efforts to ensure that women and men with disabilities can fully exercise their electoral rights by developing and disseminating standards on accessible election facilities and campaign material	F
Raise awareness among MS of the need to improve accessibility of courts and police buildings	N
Promote the dissemination of good practices regarding training of public officials on receiving and informing persons with disabilities	F
Raise awareness among MS on the need for proper assistance regarding access to legal documents and procedures	F
Explore ways of facilitating the use of sign language in dealing with the EU institutions	F
Monitor the proper implementation of article 82(2) of the TFEU (rights of individuals in criminal procedure) relative to interpretation and translation in criminal proceedings on the specific issue of interpretation in sign language	F
Ensure inclusion of concerns of person with disabilities in initiatives aimed to address consumer rights in Europe, in particular with regard to access to services of general economic interest	F
Address problems related to intra-EU mobility
Promote dialogue among Member States in the Disability High Level Group on the portability of rights such as the right to personal assistance	F
Encourage the participation of the EU Member States to the 2000 Hague Convention on the International Protection of Adults and evaluate its application, foreseen in the Stockholm Action Plan, with the view to assess the need for additional proposals as regards vulnerable adults	P
Identify obstacles encountered by persons with disabilities in the exercise of their rights as EU citizens, in particular regarding the right to free movement and residence	F
Enhance the use of the European model of disability parking card
Raise awareness of the European model of disability parking card and combat its abuse by producing and disseminating information material for national and local authorities	F
Explore opportunities of new technological solutions, for example by funding pilot projects on the development of electronic parking cards and control systems	F
Enhance Member States' efforts towards the transition from institutional to community based care
Develop and disseminate a quality framework for community-based services that is inclusive of person with disabilities building on the quality framework for Social services of general interest	F
Promote the exchange of good practices among Member States in the Disability High Level Group on personal assistance funding schemes	F
Optimise the use of Structural Funds and the Rural Development Fund to support the development of community-based services
Identify good practice of the use of Structural Funds and the Rural Development Fund in relation with the principle of independent and community living	P
Develop a training module for European Commission geographical Desk officers (and national administrations) dealing with Structural Funds and the Rural Development Fund on the UNCRPD and common basic principles on deinstitutionalisation	F
Develop a toolkit for managing authorities on how to use Structural Funds and the Rural Development Fund to support the development of community-based services	F
Report on Member States' compliance with article 16 of Structural funds general regulation	F
Improve the knowledge base on the situation of people with disabilities living in residential institutions
Collect data on the number, size and life conditions of residential institutions	P
Study on existing legal and administrative rules which directly or indirectly promote institutionalisation with recommendations on how to remove them	P
Promote the participation of people with disabilities in sports
Develop and disseminate standards for accessibility of sports, leisure, and recreation organisations, activities, events and venues	F
Promote the participation of people with disabilities in European sport events as well as the organisation of disability-specific events including Special Olympics	F
Include a priority on Social inclusion through and in sport, with a particular regard to persons with disabilities, in the future Commission policy and incentive measures in the field of sport	F
Promote the access of people with disabilities to cultural materials and events
Foster the cross-border transfer of copyright works in accessible format	F
Ensure accessibility of the interface and contents of Europeana – the European public digital library – for persons with disabilities	F
Ensure that accessibility criteria are taken into account in the context of the "European Capitals of Culture" award	F

Source: ICF analysis

EU level actions associated with Equality

The equality area of action aimed to combat discrimination based on disability and promote equal opportunities. The implementation status of EU level actions is outlined in the table below

Table 6: implementation of actions in the area of equality

Promote and protect the inherent dignity of persons with disabilities, combat all forms of discrimination on the basis of disability, and ensure that persons with disabilities enjoy, on an equal basis with others, all fundamental rights and freedom
Support the negotiation in Council of the proposal for the Directive on equal treatment beyond the field of employment	P
Monitoring the application and impact of Directive 2000/78 EC for improving employment of persons with disabilities	F
Promote the attention to disability matters in equality bodies notably through Equinet	F
Provide guidance on reasonable accommodation for people with disabilities addressing employers and service providers	F
Promote exchange of good practices on legal capacity	F
Raise awareness among trade unions on the concept of reasonable accommodation	P
Use of existing Progress programme (until December 2013) to support national activities aiming at combating discrimination and promoting equality	F
Address disability issues in awareness raising seminars in the areas of non-discrimination and equality targeted at civil society organisations	F
Address disability discrimination in annual calls for proposals aiming at supporting national authorities in their fight against discrimination and promotion of equality	F
Introduce a disability specific focus in the "What can Social Europe do for you" campaign	F
Address disability discrimination in the "For Diversity Against Discrimination campaign", e.g. through Diversity days, Journalist award, leaflet, video spot, stand at conferences	F

Source: ICF analysis

EU level actions associated with Employment

The employment area of action aimed to increase the participation of people with disabilities in the labour market, where they are currently under-represented. The implementation status of EU level actions is outlined in the table below

Table 7: implementation of actions in the area of employment

Increase knowledge of employment situation of people with disabilities, identify challenges, propose remedies
Use 2011 LFS ad hoc module to produce information materials	F
Enhance cooperation with relevant third organisations	F
Explore a stronger focus on women and men with disabilities	F
Optimise the use of the new strategy for jobs and growth, ‘Europe 2020’, for the benefit of people with disabilities
Ensure annual Europe 2020 assessment considers people with disabilities	P
Preparation of disability indicator in EMCO indicator subgroup	N
Make use of ESIF to promote labour market integration of people with disabilities and other vulnerable groups	F
Focus on what people can do and persuade potential employers through convincing arguments and support to people with disabilities
Involve social partners at the EU level	P
When reviewing European employment legislation ensure compliance with UNCPRD	P
Encourage employers to commit to diversity policies	F
Highlight the scope for action Member States enjoy through the GBER	F
Encourage social entrepreneurship with concrete actions in a ‘Social Business Initiative’	F
Give special attention to the difficulties of young people with disabilities in the transition from education to employment and address intra-job mobility, including those working in sheltered workshops (access to and retention in employment)
Involvement in PES at the EU level	F
Identify and promote effective support structures	F
Address the issue of quality of jobs, such as salaries, working hours and career advancement of people with disabilities
Promote labour market relevant training to people with disabilities	F
Foster possibilities of self-employment opportunities for people with disabilities	F
Fight prevailing disability benefit cultures and help to integrate persons with partial work capacity into the labour market, further develop ALMPs and tackle benefit traps
Examine national ALMPs and publish good practices	F

Source: ICF analysis

EU level actions associated with Education and training

The education and training area of action aimed to promote inclusive education and lifelong learning for students and pupils with disabilities. Equal access to quality education and lifelong learning enable disabled people to participate fully in society and improve their quality of life. The implementation status of EU level actions is outlined in the table below.

Table 8: implementation of actions in the area of education and training

Increase knowledge on education levels and opportunities of people with disabilities
Promote peer reviews on the different definitions at Member State level of inclusive education through the support of the Network of Experts on the Social Sciences of Education and Training (NESSE)	P
Support the work of the European Agency for Development in Special Needs Education to facilitate the collection, processing and transfer of European level and country specific information on education of people with disabilities	P
Collect data on access and participation of persons with high dependency needs in the education system	P
Improve e-skills of persons with disabilities	F
Support policy developments towards the goal of inclusive and quality education and training within the framework of the Youth on the move initiative
Promote the exchange of good practice on inclusive education and lifelong learning for students and pupils with disabilities within the strategic framework for European cooperation in education and training ET 2020	F
Raise disability matters in the context of the OMC on education, with due attention to the specific needs of women and men with disabilities	P
Disseminate research, information, and guidance materials on the application of the reasonable accommodation principle in education and training	P
Monitor recent developments regarding national and/or Europe-wide Curricula for professionals in the built environment, transport and ICT on Design for all to improve their knowledge, skills and competences on accessibility and encourage the development of a European Curriculum	N
Increase the mobility of people with disabilities through enhancing their participation in the Lifelong Learning Programme and the Youth in Action Programme
Ensure that relevant calls for proposals under the Lifelong Learning programme include accessibility and reasonable accommodation criteria	F
Maintain the priority given to young people with fewer opportunities (including young people with disabilities) in the implementation of the Youth in Action Programme	F
Promote life long learning of people with disabilities working in the police or justice systems	P

Source: ICF analysis

EU level actions associated with Social protection

The social protection area of action aimed to promote decent living conditions, combat poverty, and social exclusion. The implementation status of EU level actions is outlined in the table below.

Table 9: implementation of actions in the area of social protection

Optimise the use of the European Platform against Poverty and the ESF
Promote cooperation, peer review and good practice exchange on disability issues, with due attention to gender differences	P
Promote the design and implementation of social innovation programmes for persons with disabilities	F
Reduce social exclusion by targeted support of concrete actions through the ESF under the framework of the European Platform against poverty	F
Assess the adequacy of social protection systems with respect to people with disabilities
Ensure that disability-specific issues are covered in any revisions of legislation concerning pensions and benefits portability	P
Follow up the Green Paper on pensions to take into account disability-relevant aspects, organise exchange of good practices among MS	F
Encourage Member States to address in the SPSI the situation of persons with disabilities and to take measures to compensate the financial impact of disabilities	F

Source: ICF analysis

EU level actions associated with Health

The health area of action aimed to promote equal access to health services and related facilities. The implementation status of EU level actions is outlined in the table below

Table 10: implementation of actions in the area of health

Support policy developments to improve equal access to healthcare
Develop indicators to monitor quality and accessibility of health care services for women and men with disabilities involving the service users’ perspective	P
Promote equal access to health care systems and raise awareness among persons with disabilities of their rights of access	P
Raise disability awareness and specific knowledge among health professionals, considering the specific needs of female and male patients	P
Support policy developments to improve quality of healthcare and rehabilitation for people with disabilities
Explore the possibility of including disability information in the e-health medical records and ensure their accessibility for persons with disabilities	F
Develop accessibility standards for medical equipment	P
Support research on healthcare provision to women and men with disabilities through health work programmes in FP7 and FP8	F
Exploit the potential of new telemedicine services such as online medical consultations, improved emergency, care and portable devices for persons with disabilities	F
Promote modern mental health services and long-term care facilities through the current European Pact for Mental Health and Well-being	F
Promote training of health professionals on disability matters through the ESF	N
Promote actions in the field of health and safety at work to reduce risks of disability during working life
Examine follow up of the specific action taken by Member States to improve the rehabilitation and reintegration of workers excluded from the workplace for a long period of time because of an accident at work, an occupational illness or a disability in the mid-term review of the EU Strategy on Health and Safety at Work 2007-2012 Report on the implementation of the European social partners' framework agreement on work-related stress	F
Explore the possibility to address the needs of people acquiring a disability while working at sea, particularly in the context of the revision of financial instruments	F
Address the issue of prevention of disabilities including work related disabilities for reasons of mental disabilities	F

Source: ICF analysis

EU level actions associated with External Action

The external action area of action aimed to promote the rights of people with disabilities in the EU enlargement and international development programmes. The implementation status of EU level actions is outlined in the table below

Table 11: implementation of actions in the area of external action

Promote the rights of people with disabilities within a broader non-discriminatory approach in the EU external action, including the enlargement process and development programmes, taking due account of the common EU and Member States' approach to development

Ensure that the specific needs of persons with disabilities, including those who are disabled as a consequence of natural and man-made disasters, are properly assessed and addressed in the area of emergency and humanitarian aid outside the EU	P
Highlight disability where appropriate as a human rights issue in the EU human rights dialogues with third countries, based on the principles of the CRPD	F
Ensure that EU development cooperation reaches persons with disabilities, both through projects/programmes specifically targeting persons with disabilities and by improving the mainstreaming of disability concerns	P
Enhance coherence and complementarity between the EU approach to persons with disabilities and EU assistance to survivors of landmine and explosive remnants of war during armed conflict and its aftermath	P
Support the national efforts of partner countries for the signature, ratification and implementation of the CRPD	P
Support where appropriate the institutional strengthening of Disabled Peoples' Organisations in partner countries and organisations dealing with disability and development	P
Consolidate the network of disability focal points in EU Delegations and at Headquarters, and increase the awareness of EU staff on disability matters	P
Promote that infrastructure financed in the framework of EU development projects meets the accessibility requirements of people with disabilities.	P
Update the Guidance Note on Disability and Development to be in line with the UN Convention on the Rights of Person with Disabilities	F
Ensure that progress is made by candidate and potential candidate countries on promoting the rights of persons with disabilities and that accession funds are used to improve their situation	P
Encourage European Neighbourhood Policy partner countries to protect the rights of persons with disabilities; mainstream disability-related issues through policy dialogue and exchange of experience; and explore possibilities for financial assistance under the European Neighbourhood Policy and Partnership Instrument	P

Source: ICF analysis

Annex 6: Evaluation questions

Effectiveness

-To what extent have the objectives set out in the Strategy been achieved overall and for each main area of action and for the implementation instruments?

-To what extent have the actions defined under the EU Disability Strategy been implemented at the EU level for each main area of action and the implementation instruments?

-To what extent did the EU Disability Strategy mainstream disability issues in EU policy and legislation?

-To what extent did the EU Disability Strategy contribute to the implementation of the UN Convention and in setting up governance frameworks both at EU level and within the EU institutions (mechanisms under article 33 of the CRPD)?

-What have been success factors in implementing the EU Strategy? Why?

-What have been possible gaps or challenges that have hindered achievement of objectives?

-To what extent have stakeholders been actively engaged in the strategy’s implementation and how have they been affected?

Efficiency

-To what extent has the EU Disability Strategy been cost effective?

-To what extent were the resources (and especially EU funding and national match funding) across the areas of action at the EU level adequate and proportionate?

Relevance

-To what extent have the EU Disability Strategy policy process and objectives been instrumental – and continue to be relevant considering evolving policy context (in view of the CRPD, European Pillar of Social Rights and European Semester) in addressing the needs of persons with disabilities?

-To what extent is the EU Disability Strategy (still) relevant for its different stakeholders, including European citizens in general?

Coherence

-To what extent is the EU Disability Strategy coherent internally, i.e. how do selected thematic areas and implementation instruments of the Strategy work together?

-To what extent is the EU Disability Strategy coherent with the EU policies/ actions, notably Europe 2020, the European Pillar of Social Rights, the European Semester, the Charter of Fundamental Rights of the European Union and the EU funding provisions? To what extent was disability mainstreamed in those policies/actions?

-To what extent is the EU Disability Strategy coherent with the CRPD?

-To what extent have the EU level actions for each area been coherent with the related policy measures in Member States?

Added value

-What is the added value of the EU strategy in eliminating barriers for persons with disabilities and in implementing the CRPD compared to what is likely to have been achieved both at the EU level (including by institutions) and MS levels in its absence?

Annex 7: Lists of figures and tables

Figures included in the main document

Figure 1: Implementation of the Strategy overall

Figures 2.1 and 2.2: Implementation by thematic area (number and percentage)

Figures 3.1 and 3.2: Implementation by implementation instrument (number and percentage)

Figure 4: Relevance of the current areas of action for future disability strategy

Figure 5: Alignment between national priority measures and the areas of action of the Strategy

Figure 6: Extent to which the Strategy has delivered added value in implementing the UNCRPD

Figure 7: Role of EU initiatives in improving the situation of persons with disabilities in the last ten years

Tables included in the main document

Table 1: Views on whether the situation for persons with disabilities improved in the Member States over the last ten year s

Table 2: Familiarity with the Strategy in the Member States

Table 3: Examples of Member State policies and comparison with the objectives of the Strategy

Figures included in the annexes

Figure 1: Consultation methods and their target groups

Figure 2: Public consultation – respondents per country of origin

Figure 3: Public consultation – important topics identified by respondents

Tables included in the annexes

Table 1: Public consultation - effectiveness results for the different areas of the Strategy

Table 2: EU-level stakeholders interviewed

Table 3: Judgement criteria about the implementation of the actions

Table 4: implementation of actions in the area of accessibility

Table 5: implementation of actions in the area of participation

Table 6: implementation of actions in the area of equality

Table 7: implementation of actions in the area of employment

Table 8: implementation of actions in the area of education and training

Table 9: implementation of actions in the area of social protection

Table 10: implementation of actions in the area of health

Table 11: implementation of actions in the area of external action

(1) http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM%3A2010%3A0636%3AFIN%3Aen%3APDF

(2) The purpose of an evaluation Staff Working Document is described in detail in the Commission’s Better Regulation Guidelines (Tool #49 the evaluation Staff Working Document). The guidelines state that “the conclusions of the Staff Working Document should be written in such a way that policy makers can use them as a basis for future policy development but should not make any commitment for future action or direction of action”. As a result, an evaluation Staff Working Document includes conclusions about the performance of the initiative being evaluated, but it does not contain recommendations or suggestions on future options or measures.

(3) https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html

(4) Study supporting the evaluation of the European disability strategy (2010-2020), ICF consortium for the European Commission, 2019

(5) This exercise might not have been exhaustive and it does not prejudge in any case the implementation assessment conducted within the evaluation.

(6) Better Regulation Guidelines, SWD (2017) 350

(7) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A12012E%2FTXT

(8) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT

(9) Community Social Action Programme on the Social Integration of Handicapped People, 1983-88, (1981); HELIOS I (Second) Community Social Action Programme for Disabled People (1988) OJ L104/38; HELIOS II (Third) Community Action Programme to Assist Disabled People (1993) OJ L56/30

(10) https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2003:0650:FIN:EN:PDF

(11) p.7 of the evaluation report

(12) https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52010DC0636&from=EN

(13) https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52010SC1324&from=EN

(14) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=LEGISSUM%3Aai0020

(15) Council decision concerning the conclusion, by the European Community, of the United Nations Convention on the Rights of Persons with Disabilities (2010/48/EC); https://eur-lex.europa.eu/legal-content/FR/TXT/?uri=uriserv:OJ.L_.2010.023.01.0035.01.ENG

(16) Declaration concerning the competence of the European Community with regard to matters governed by the United Nations Convention on the Rights of Persons with Disabilities

(17) Directive (EU) 2019/882 of the European Parliament and of the Council of 17 April 2019 on the accessibility requirements for products and services; https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L0882

(18) Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies; https://eur-lex.europa.eu/eli/dir/2016/2102/oj

(19) Directive (EU) 2018/1808 of the European Parliament and of the Council of 14 November 2018 amending Directive 2010/13/EU on the coordination of certain provisions laid down by law, regulation or administrative action in Member States concerning the provision of audiovisual media services (Audiovisual Media Services Directive) in view of changing market realities; https://eur-lex.europa.eu/eli/dir/2018/1808/oj

(20) Directive (EU) 2018/1972 of the European Parliament and of the Council of 11 December 2018 establishing the European Electronic Communications Code (Recast); https://eur-lex.europa.eu/eli/dir/2018/1972/oj

(21) Results of the annual surveys from 2006 through 2008 of the Eurostat Statistics on Income, Social Inclusion and Living Conditions (SILC) and evidence from the 2002 Eurostat ad hoc disability module of the Labour Force Survey (LFS AHM).

(22) Commission Staff Working Document SEC(2010) 1323 final accompanying the communication on the European disability strategy 2010-2020 – p. 6.

(23) Commission Staff Working Document SEC(2010) 1323 final accompanying the communication on the European disability strategy 2010-2020 – Section 1.

(24) https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=SEC:2010:1324:FIN:en:PDF

(25) https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52010SC1324&from=EN

(26) http://ec.europa.eu/social/BlobServlet?docId=16995&langId=en

(27) Annex 1 of the Study supporting the evaluation of the European Disability Strategy (2010-2020), ICF consortium for the European Commission, 2019, provides details on the evaluation framework that was used to answer the evaluation questions.

(28) https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines-evaluation-fitness-checks.pdf

(29) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1929-European-Disability-Strategy-2010-2020-evaluation

(30) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1929-European-Disability-Strategy-2010-2020-evaluation

(31) More details over gaps and challenges hindering achievement of objectives are provided in Section 5.1.5

(32) Directive (EU) 2019/882 of the European Parliament and of the Council of 17 April 2019 on the accessibility requirements for products and services (Text with EEA relevance); https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L0882

(33) Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies (Text with EEA relevance ); https://eur-lex.europa.eu/eli/dir/2016/2102/oj

(34) Regulation (EU) No 1177/2010 of the European Parliament and of the Council of 24 November 2010 concerning the rights of passengers when travelling by sea and inland waterway and amending Regulation (EC) No 2006/2004 (Text with EEA relevance); https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32010R1177 . Regulation (EU) No 181/2011 of the European Parliament and of the Council of 16 February 2011 concerning the rights of passengers in bus and coach transport and amending Regulation (EC) No 2006/2004 (Text with EEA relevance); https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=celex:32011R0181

(35) Proposal for a regulation of the European Parliament and of the Council on rail passengers’ rights and obligations (recast); https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52017PC0548

(36) Commission Regulation (EU) No 1300/2014 of 18 November 2014 on the technical specifications for interoperability relating to accessibility of the Union's rail system for persons with disabilities and persons with reduced mobility (Text with EEA relevance); https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.356.01.0110.01.ENG

(37) Directive (EU) 2018/1808 of the European Parliament and of the Council of 14 November 2018 amending Directive 2010/13/EU on the coordination of certain provisions laid down by law, regulation or administrative action in Member States concerning the provision of audiovisual media services (Audiovisual Media Services Directive) in view of changing market realities; https://eur-lex.europa.eu/eli/dir/2018/1808/oj

(38) Directive (EU) 2018/1972 of the European Parliament and of the Council of 11 December 2018 establishing the European Electronic Communications Code (Recast) (Text with EEA relevance); https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2018.321.01.0036.01.ENG

(39) Directive (EU) 2017/1564 on certain permitted uses of certain works and other subject matter protected by copyright and related rights for the benefit of persons who are blind, visually impaired or otherwise print-disabled; https://eur-lex.europa.eu/eli/dir/2017/1564/oj . The Directive is implementing the “Marrakesh Treaty to Facilitate Access to Published Works for Persons Who Are Blind, Visually Impaired, or Otherwise Print Disabled”, which aims to facilitate access to print works in formats adapted for persons who are blind, visually impaired or otherwise print disabled. The Marrakesh Treaty, to which the EU is party, is an international copyright treaty administered by the World Intellectual Property Organisation (WIPO).

(40) See Art. 17 of Regulation (EU) No 181/2011 for bus and coach, Art. 15 of Regulation (EU) No 1177/2010 for sea and inland waterways, Art. 25 of Regulation (EC) 1371/2007 for rail (which will be further strengthened by the new recast rules, see above). In relation to air, the Commission proposed in 2013 that the liability of air carriers with regard to mobility equipment will be increased up to the actual value of the equipment (see COM(2013)130 final, draft Article 6a of the amended Regulation (EC) No 2027/97; https://eur-lex.europa.eu/procedure/EN/2013_72 ). The procedure is still ongoing and the proposal is a priority pending file in the Commission Work Programme 2021 (see COM(2020) 690 final, Annex III number 5; https://ec.europa.eu/info/publications/2021-commission-work-programme-key-documents_en )

(41) However, progress was registered more recently in relation to the adoption of European Accessibility Standard EN 17210 linked to Standardisation Mandate 420 on accessibility to the built environment. The European Committee for Standardisation has announced the ratification of the standard for 30 November 2020 and its publication for January 2021. More information is available at https://standards.cen.eu/dyn/www/f?p=204:110:0::::FSP_PROJECT,FSP_LANG_ID:65077,25&cs=1B1F504D7DCF7711690E22BAE7CED456A and https://www.cencenelec.eu/standards/Topics/Accessibility/Pages/Builtenvironment.aspx

(42)

Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013 laying down common provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund and laying down general provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund and the European Maritime and Fisheries Fund and repealing Council Regulation (EC) No 1083/2006; https://eur-lex.europa.eu/legal-content/en/TXT/?uri=celex%3A32013R1303

(43)

Ex-ante conditionality 9.1 on the existence and the implementation of a national strategic policy framework for poverty reduction aiming at the active inclusion of people excluded from the labour market in the light of the Employment Guidelines covering the measures for the shift from institutional to community- based care; General ex-ante conditionality No 3 on the existence of administrative capacity for the implementation and application of the United Nations Convention on the rights of persons with disabilities (UNCRPD) in the field of ESI Funds in accordance with Council Decision 2010/48/EC.

(44) https://ec.europa.eu/social/main.jsp?catId=1139

(45) Belgium, Cyprus, Estonia, Finland, Italy, Malta, Romania and Slovenia

(46) https://ec.europa.eu/assets/eac/youth/library/reports/inclusion-diversity-strategy_en.pdf

(47) https://ec.europa.eu/programmes/erasmus-plus/projects_en#search/project/keyword=disabilities&matchAllCountries=false

(48) https://ec.europa.eu/sport/about/initiatives/beinclusive_en

(49) https://ec.europa.eu/social/main.jsp?langId=en&catId=88&eventsId=1487&furtherEvents=yes

(50) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52019XG0607(03)

(51) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32000L0078

(52) REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Joint Report on the application of Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin (‘Racial Equality Directive’) and of Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation (‘Employment Equality Directive’) COM/2014/02 final; https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52014DC0002

(53) https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52008PC0426&from=DA

(54) https://ec.europa.eu/commission/priorities/deeper-and-fairer-economic-and-monetary-union/european-pillar-social-rights/european-pillar-social-rights-20-principles_en

(55) https://ec.europa.eu/eurostat/web/lfs/overview

(56) https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32010R0317

(57) ANED ( https://www.disability-europe.net/ ) was created by the European Commission in December 2007 to support policy development through studies, analysis, research and publications. It has been replaced by the European Disability Expertise project in 2020.

(58) Eurostat EU SILC data analysed by ANED.

(59) Eurostat EU SILC data analysed by ANED

(60) https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=hlth_dpe010&lang=en

(61) https://ec.europa.eu/commission/sites/beta-political/files/social-summit-european-pillar-social-rights-booklet_en.pdf

(62) Directive (EU) 2019/1158 of the European Parliament and of the Council of 20 June 2019 on work-life balance for parents and carers and repealing Council Directive 2010/18/EU; https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2019.188.01.0079.01.ENG

(63)

Council Recommendation of 8 November 2019 on access to social protection for workers and the self-employed 2019/C 387/01 ST/12753/2019/INIT; https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.C_.2019.387.01.0001.01.ENG&toc=OJ:C:2019:387:TOC

(64) https://ec.europa.eu/esf/home.jsp

(65) https://ec.europa.eu/justice/grants1/programmes-2014-2020/rec/index_en.htm

(66) https://ec.europa.eu/social/main.jsp?catId=1081

(67) https://ec.europa.eu/info/funding-tenders/funding-opportunities/funding-programmes/overview-funding-programmes/structural-reform-support-programme-srsp_en

(68) Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the application of patients’ rights in cross-border healthcare; https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32011L0024

(69) https://ec.europa.eu/eurostat/web/lfs/overview

(70) https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32010R0317

(71) https://ec.europa.eu/eurostat/web/microdata/european-health-interview-survey

(72) https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Disability_statistics

(73) EUROSTAT table [HLTH_DH030], dataset based on EU-SILC ;

(74) https://ec.europa.eu/anti-trafficking/eu-policy/action-plan-human-rights-and-democracy-2015-2019_en

(75) DG ECHO Operational Guidance - The Inclusion of Persons with Disabilities in EU-funded Humanitarian Aid Operations

(76) https://ec.europa.eu/international-partnerships/european-consensus-development_en

(77) http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%209489%202014%20INIT

(78) https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=DCD/DAC/STAT(2018)39/REV1&docLanguage=En

(79) https://www.washingtongroup-disability.com/question-sets/wg-short-set-on-functioning-wg-ss/

(80) ec.europa.eu/social/accesscityaward

(81) European Day of Persons with Disabilities 2019: https://ec.europa.eu/social/main.jsp?langId=en&catId=88&eventsId=1487&furtherEvents=yes

(82) https://ec.europa.eu/social/main.jsp?langId=en&catId=1141&eventsId=1442&furtherEvents=yes

(83) https://ec.europa.eu/social/euvsdiscrimination

(84) Eurostat population surveys: Labour Force Survey (LFS) , Statistics on Income and Living Conditions (SILC) , European Health Interview Survey (EHIS) , Adult Education Survey (AES ), Information and Communication Technologies survey (ICT ), Time Use Survey (TUS ), Household Budget Survey (HBS)

(85) https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities/article-33-national-implementation-and-monitoring.html

(86) https://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-%20Europe%202020%20-%20EN%20version.pdf

(87) https://ec.europa.eu/commission/priorities/deeper-and-fairer-economic-and-monetary-union/european-pillar-social-rights/european-pillar-social-rights-20-principles_en

(88) The European Semester is a cycle of economic and fiscal policy coordination within the EU. It is part of the European Union's economic governance framework. The cycle takes place for a 6-month period from the beginning of each year, hence its name - the 'semester'. During the European Semester, the Member States align their budgetary and economic policies with the objectives and rules agreed at the EU level.

(89) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52007DC0630

(90) https://ec.europa.eu/digital-single-market/en/content/european-digital-strategy

(91) https://op.europa.eu/en/publication-detail/-/publication/8e5d0991-82ed-11e5-b8b7-01aa75ed71a1 and https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52012XG0308%2801%29

(92) https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A52011DC0144

(93) https://ec.europa.eu/youth/policy/youth-strategy_en

(94) https://ec.europa.eu/anti-trafficking/eu-policy/strategic-engagement-gender-equality-2016-2019_en

(95) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2020:628:FIN

(96) https://ec.europa.eu/education/resources-and-tools/document-library/eea-communication-sept2020_en

(97) https://ec.europa.eu/education/sites/education/files/document-library-docs/deap-communication-sept2020_en.pdf

(98) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52020DC0152

(99) https://ec.europa.eu/info/sites/info/files/a_union_of_equality_eu_action_plan_against_racism_2020_-2025_en.pdf

(100) The principle states that “People with disabilities have the right to income support that ensures living in dignity, services that enable them to participate in the labour market and in society, and a work environment adapted to their needs”.

(101) https://intragate.ec.europa.eu/decide/sep/index-compressed.html?Zebra_v3.14.7.15803-2020-04-16T15:23:39.817+02:00#/overview-screen/view=dossier-details&planning-id=SPECIFIC-DOSSIER-2018-36357

(102) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1929-European-Disability-Strategy-2010-2020-evaluation

(103) https://www.disability-europe.net/

(104) https://ec.europa.eu/social/main.jsp?langId=en&catId=88&eventsId=1487&furtherEvents=yes

(105) https://www.eesc.europa.eu/en/our-work/opinions-information-reports/opinions/shaping-eu-agenda-disability-rights-2020-2030-contribution-european-economic-and-social-committee-own-initiative-opinion

(106) https://www.europarl.europa.eu/doceo/document/B-9-2020-0123_EN.html

(107) https://ec.europa.eu/eurostat/web/income-and-living-conditions

(108) ICF consortium for the European Commission, 2019

(109) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1929-European-Disability-Strategy-2010-2020-evaluation