EUROPEAN COMMISSION

Brussels, 1.7.2020

SWD(2020) 121 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

of the Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning

Accompanying the document

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

European Skills Agenda for sustainable competitiveness, social fairness and resilience

{COM(2020) 274 final} - {SWD(2020) 122 final}

Table of contents

1.Introduction

Purpose and scope    

2.Background to the intervention

Description of the intervention and its objectives    

3.Implementation / State of Play

3.1 Baseline and points of comparison    

3.2 Description of the current situation    

4.Method

Short description of methodology    

Limitations and robustness of findings    

5.Analysis and answers to the evaluation questions

5.1 Effectiveness    

5.2 Efficiency    

5.3 Relevance    

5.4 Coherence    

5.5 EU added value    

6.Conclusions and lessons learned

6.1 Conclusions    

6.2 Lessons learned    

Annex 1: Procedural information

Annex 2: Stakeholder consultation

Annex 3: Methods and analytical models

Glossary

1.Introduction

Purpose and scope

This document evaluates the Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning 1 (hereafter “the Recommendation”). Evaluating the implementation and impact of EU legislation is a basic requirement of the Better Regulation framework. The Recommendation itself asks the Commission to report to the Council assessing and evaluating “the action taken in response to its provisions and on the experience gained and implications for the future”. The results of the evaluation will feed into the policy debate to put in practice the European Pillar of Social Rights, namely the first and fourth principles 2 , with particular reference to the updated Skills Agenda (COM(2020) 274).

In compliance with the Better Regulation Guidelines, the evaluation is based on the following criteria:

–effectiveness: to what extent the objectives of the Recommendation have been achieved through action by Member States and the Commission;

–efficiency: relationship between costs and benefits, relevant factors and proportionality of costs;

–relevance: whether six years after the adoption of the objectives of the Recommendation, the measures it proposes to achieve them and the governance and support structures are still relevant;

–coherence: whether the Recommendation remains coherent with other EU and national initiatives;

–EU added value: whether the Recommendation has generated additional value compared to what action at national level alone would have produced.

The Recommendation asked Member States to take action no later than 2018, therefore the evaluation covers the period from the adoption of the Recommendation (end of 2012) to 2018 included.

2.Background to the intervention

Description of the intervention and its objectives

The Recommendation addresses an important element of lifelong learning. Its objective is to support all people to develop their personal and professional skills and get them recognised, so that they can play an active role in society and the labour market.

The lifelong learning approach focuses on the knowledge and skills that people develop throughout their life in any learning setting, either formal (e.g. institutional education and training programmes), non-formal (e.g. in-company training) or informal (e.g. work and life experience). The Recommendation should therefore be seen in the European policy context defined by initiatives such as the European area of lifelong learning 3 , the strategic framework for European cooperation in education and training (Education and Training 2020) 4 and more recently the 2016 Skills Agenda 5 , with particular reference to the Upskilling Pathways initiative 6 , and the European Education Area 7 . Its implementation is related with the use of other tools developed within this policy process, such as Europass 8 , the European Qualifications Framework (EQF) 9 , the European Credit System for Vocational Education Training (ECVET) 10 and contributes to putting in practice the first and fourth principles of the European Pillar of Social Rights.

Impact assessment 11 . In this context, a Council Recommendation on validation of non-formal and informal learning was considered appropriate to address the following problems identified through the impact assessment: while there was widespread consensus on the benefits of skills validation for individuals, the economy, and society, in most Member States people had limited opportunities to validate their skills and where such opportunities were available, their use was limited. Besides, people who had their skills validated in one country might not be able to use the results in another Member State, as the differences in approach, scope and conceptual framework made it difficult to compare validation results across countries.

Addressing these problems meant pursuing the general objectives of providing individuals in all Member States with opportunities to validate their skills acquired outside formal education and training systems and to use the results of validation to study or work anywhere in Europe 12 .

Figure 2.1. The two general objectives

Two specific objectives were defined. First, establishing of national arrangements for the validation of non formal and informal learning, linked to the national qualifications framework. Second, making sure that such national arrangements had an appropriate level of quality and coherence, so that their results could be more easily understood and compared across countries, thanks in particular to the link to the EQF.

Figure 2.2. The two specific objectives

Five operational objectives indicated the action to take to pursue the specific and general objectives. They focused on national qualifications frameworks, validation mechanisms, the value of skills identification and documentation, the need for cooperation, and awareness raising.

Figure 2.3. The five operational objectives

Recommendation 13 . As adopted by the Council, the Recommendation does not explicitly mentions the objectives of the intervention logic, but its provisions relate to them. In its first invitation the Recommendation does invite Member States to allow individuals who developed skills through non-formal and informal learning to have them validated (cf. first general objective) and use them for their careers and further learning (cf. second general objective). For this purpose, Member States should put in place, by 2018, arrangements for the validation of non-formal and informal learning, which may give priority to certain areas or sectors, so that people may eventually obtain through validation a formal qualification or credit towards it (Invitation 1.1, referring to specific objective 1).

People should be able to take advantage of each of the four phases of validation – identification, documentation, assessment and certification – separately or in a coordinated process, so that individual validation pathways are flexible and adapted to specific individual needs (Invitation 1.2 to Member States, related to operational objectives 2 and 3).

The Recommendation also provides a number of principles to be applied in setting up validation arrangements (Invitation 1.3 to Member States). They concern the link to national qualifications frameworks and the EQF (specific objective 1), the role of information and guidance (operational objective 5), qualification standards and use of learning outcomes (operational objective 1), the use of European transparency documents (specific objective 2), as well as specific target groups, quality assurance, skills audits, credit systems and validation staff – all complementary elements to pursue the various objectives. Invitation 1.4 to Member States focuses on cooperation among stakeholders (operational objective 4).

The Recommendation asks the Commission to organise peer learning activities (operational objective 4) and support the national efforts by updating the European guidelines for validation of non-formal and informal learning 14 and the European Inventory of validation of non-formal and informal learning 15 (hereafter “Inventory”). As concerns governance, the Recommendation indicates the EQF advisory group, set up following the 2008 EQF Recommendation and confirmed by the 2017 EQF Recommendation, as the policy body most appropriate for its follow up.

The Recommendation emphasises that validation promotes mobility (recitals 1 and 2), and Invitation 1 to Member States refers to the opportunity for people to use their validated skills for their careers and further learning (referring to general objective 1). Supporting people to use the results of validation across countries (second general objective) is pursued through European transparency tools: if validation arrangements are properly linked to national qualifications frameworks aligned to the EQF (Invitation 1.3.a, pursuing second specific objective and the first operational objective), then validation leads to qualifications, or parts of them, which are easier to understand and compare across Europe. Skills that are documented, but not assessed and certified in the form of qualifications (cf. operational objective 3 and Invitation 1.2), can still be made transparent in all Member States through the use of such instruments as the Youthpass or the Europass Mobility (cf. Invitation 1.3.i).

In its recitals and definitions, the Recommendation stresses that validation of non-formal and informal learning is a transversal policy area, which contributes to implementing policies related to education and training, youth, employment and social inclusion. Validation is part and parcel of the lifelong learning approach and the shift to learning outcomes – knowledge, skills and competences acquired throughout life in a variety of settings, from school to work experience to volunteering and open learning resources. Given its cross-sector nature, implementing validation requires coordinated action by many institutional, social and economic actors. As a direct result, validation is found embedded in much of the EU policy on vocational education and training 16 , higher education 17 , adult learning (cf. Upskilling Pathways), school education 18 , employment policy 19 and youth policy 20 . The figure below summarises the intervention logic.

Figure 2.4. Intervention logic

3.Implementation / State of Play

3.1 Baseline and points of comparison

In 2012 21 , some validation arrangements were in place in 20 Member States, with huge differences in scope and stage of development. Four Member States had well-established validation arrangements leading to qualifications (Finland, France, the Netherlands and Portugal) 22 . France was the only country where all qualifications in the national qualifications directory (with the exception of a few regulated professions) could also be obtained on the basis of validation of non-formal and informal learning. Seven further countries (Denmark, Germany, Luxembourg, Romania, Spain, Sweden and the United Kingdom) had either a national system in its initial phase or a well-established, but partial, system of validation in one or more areas. Limited arrangements were also available in Austria, Belgium, the Czech Republic, Estonia, Italy, Ireland, Lithuania, Slovakia and Slovenia, e.g. in Belgium (Flanders) and Slovenia validation could support people obtaining vocational qualifications. In other Member States some opportunities for validation were also available, but mostly in the form of single initatives without any systematic character. Validation was part of the policy debate in all Member States, but only 13 Member States declared to have some form of validation strategy in place or in development 23 . Data on the take-up of validation opportunities were very scarce in 2012 24 .

The figure below summarises the situation in Member States before the adoption of the Recommendation, as concerns the availability of validation arrangements and related features described below.

Figure 3.1–1. Validation in Member States in 2012 25  

Source: Inventory 2010

In 2010 validation could lead to partial or full qualifications in 24 Member States 26 , while in six countries validation could grant access to formal education programmes 27 . The link to national qualifications frameworks (NQFs) was uncertain, as at the time NQFs were being developed in all Member States, while they were fully operational only in four Member States and were approaching operation in a further nine countries 28 . At the time 16 Member States had referenced national qualification levels to the EQF 29 . 

In 2010, the quality of validation was assured through a specific framework in five countries, while 11 others applied wider quality assurance frameworks 30 . While 24 Member States reported equivalence of standard between validation and formal education and training, that was rather a declaration of principle, as it also included countries that did not have a validation system in place. 

Often validation only concerns a limited set of skills, which would not be enough for the award of a full qualification. Validation results could however be certified in formal credit that may by further cumulated towards a full qualification. In 2010 this was possible in 10 Member States 31 .

Data from the 2010 Inventory shows that 17 Member States 32 used the four stages of validation (identification, documentation, assessment and certification of skills) as a reference. While the Recommendation had not yet been proposed, the four stages had been detailed in the European guidelines for validating non-formal and informal learning, first released in 2009.

There was little information about how guidance services supported people who might benefit from validation – e.g. conducting skill reviews, providing information on validation opportunities, and advising and assisting individuals throughout validation processes.

In 2010, skills audits 33 to facilitate validation or res-skilling of disadvantaged groups were offered in ten Member States 34 , while five countries organised specific targeted initiatives for migrants or refugees 35 .

Mandatory professional requirements (in terms of training, experience, qualification) for validation staff were reported in 17 Member States in 2010.

Country reports in the 2010 Inventory show that in 14 Member States there was a good level of involvement of most relevant stakeholders in validation activities.

3.2 Description of the current situation

3.2.1 By the end of 2018 36 , it appears that all Member States 37 have taken some action towards the objectives of the Recommendation, though not enough to make validation opportunities available to all. Some degree of validation arrangements are in place in all Member States 38 , almost all of which have or are developing some form of validation strategy 39 . Validation is still primarily connected to education and training institutions, 40 but validation opportunities have also increased in the labour market and the third sector 41 – more recent growth has especially concerned the labour market 42 . In 17 Member States there are arrangements for skills validation provided by labour market actors, often promoted by employers or associations of employers, sometimes in collaboration with public sector institutions. In 20 Member States there are validation arrangements made available in the third sector. This includes initiatives associated with youth work or volunteering, as well as validation opportunities developed by charities or other non-governmental organisations (NGOs) to support specific target groups such as third country nationals or people with a disability. Labour market actors such as Chambers of Commerce may be able to validate and certify skills, while in the third sector the focus is more on the documentation of skills 43 . While information on such validation opportunities was not available in 2012, the large share of countries concerned suggests a significant increase in opportunities in Member States, as also confirmed by a huge majority (186 out of 261) of respondents to the public consultation 44 .

While some developments took place in all Member States, in a few countries (Croatia, Hungary, Slovakia) arrangements remain quite limited and in most, they are not comprehensive. Three systems (in Belgium-Flanders, Italy and Portugal) have been developed as broad frameworks covering all areas, adding them to the wide national arrangements already in place in 2012 (in Denmark, Finland, France, Luxembourg and Sweden). All other Member States show a variety of less comprehensive approaches. Sometimes good projects are not considered up-scalable, such as the German ValiKom project 45 . Wide systems may be under development, but still focus on vocational education and training, as in Poland 46 . Different frameworks often coexist, e.g. addressing the ‘labour market route’ and the ‘education route’ as in the Netherlands 47 , or the subsectors of education and training, as is the case in Spain 48 or Lithuania 49 . All arrangements have some specific limitations. For instance, validation may only be available to workers with work experience, as is the case in the otherwise comprehensive systems in France and Luxembourg 50 . 

The figure in the next page summarises the situation of validation in Member States before the adoption of the Recommendation (cf. Figure 3.1–1) and in 2018, as concerns the availability of validation arrangements and related features.

Figure 3.2–1. Validation in Member States 2012 – 2018

Source: Inventory 2010 and 2018.

3.2.2. Some data on participation are now available from most Member States, but they are often not comprehensive, e.g. referring only to a specific sub-sector of education and training, and fragmentary 51 . However, the external study concludes, based on the 2018 Inventory triangulated with other sources, that in 14 Member States (BE, BG, CY, EL, ES, FI, IT, LU, LV, MT, PL, PT, SE, SI, SK) there has been an upward trend in take-up of validation opportunities after 2012 52 . Use remained stable in three countries, while it slightly decreased in countries (Denmark, France and the Netherlands) with a solid validation system, as well as in Romania 53 . The figure below summarises the trend in the use of validation comparing data from 2016 and 2018, also showing the lack of data for several countries.

Figure 3.2-2. Trend in the use of validation (2018 compared to 2016)

Source: Inventory 2016 and 2018. Cf. External study, Table 6.

In 22 Member States validation has become more accessible, that is people find it easier to apply 54 . This may have been achieved through embedment in labour policies (Italy), adult education policy (Estonia), equality of standards (as in Lithuania) or specific measures, such as the German ValiKom initiative  55 . Compared to the policy attention to validation and the growth of opportunities, the increase in take-up by people may appear limited. Besides, the external study found evidence of some ‘evaporation effect’: people who start a validation procedure and do not complete it 56 . In fact, engaging in validation processes requires a serious commitment by individuals 57 , and forms of active support to individuals, such as paid time made available by the employer or financial aid, are not common. Barely more than one in ten validation beneficiaries responding to the public consultation (8 out of 62, or 11%) had received an incentive or support to participate in validation 58 .

3.2.3. The increase in validation arrangements is correlated to the sustained development of national qualifications frameworks (NQFs) based on learning outcomes and their connection to validation arrangements 59 . Both developments are related to the shift towards learning outcomes in defining and describing qualifications standards, which occurred in all parts of education and training. This approach promotes the value and facilitates the assessment of skills developed outside formal education and training. By the end of 2018, almost all Member States 60 had an NQF referenced to the EQF. In 24 Member States, validation can lead to partial or full qualifications included in the NQF and in 17 it enables people to access formal education programmes leading to qualifications included in the NQF 61 . In 13 countries, any qualification included on the NQF can be acquired through validation 62 . 

3.2.4. In most Member States, validation can lead to qualifications applying the same standards used in formal programmes, although in a few countries equivalence of standards is still being developed. In many countries, there is a mixed approach, so that validation can also lead to qualifications that are not the same as those delivered through formal education and training. This equivalence – still in development in a few countries – has been strongly promoted by the gradual shift away from input based standards, restricting time, learning content and context, towards standards 7based on learning outcomes. Still, the external study points out that in some countries (BG, ES, SI) results of formal education retain a higher status and in others (EL, MT) the opportunity to obtain tertiary qualifications through validation is largely theoretical 63 . Credibility of validation results is also pursued through dedicated quality assurance frameworks, as done in 13 Member States. Five of them also use wider quality frameworks, which is the approach of other ten countries 64 . A majority of organisations responding to the public consultation (111 out of 160) agreed that validation arrangements in their country met clearly established quality standards and slightly fewer (100 out of 157) acknowledged that this led to reliable and credible results 65 .

Source: Inventory 2010 and 2018. External study, Table 4.10.

3.2.5. In 2018, formal credit towards a qualification could be awarded through validation in 22 Member States, and in about as many countries people could be granted exemption from formal programmes, more frequently in higher education 66 . However, Member States have not given priority to strong synergies between credit systems and validation. The external study has found little information on whether credit obtained through validation is converted in the European Credit Transfer System (ECTS), which might support its portability to higher education programmes in other countries 67 .

Figure 3.2–5. Validation leading to formal credit – 2010 and 2018

Source: Inventory 2010 and 2018. External study, Figure 13.

3.2.6. While “all four stages of validation are in place in the vast majority of Member States” 68 , comparison with the baseline is difficult for several reasons. First, because the information collected in the 2018 Inventory refers to the use in three areas (education and training, labour market and third sector) not applied in 2010. Second, because at national level different terminology may be used, sometimes to reflect specific practices – e.g. the phases of identification and documentation may be carried out together (Luxembourg) and be both covered by a phase called ‘counselling’ (Spain) 69 . Finally, the practical use of the stages depends on the purpose and context of each individual validation pathway. For instance, exemption from parts of a formal programme following validation may be considered or not as a stage of certification (into formal credits) based on the organisation of that specific programme and the system of which it is part. Therefore, while 16 Member States explicitly refer to the four stages as presented in the Recommendation, in others all stages are covered in validation processes, but called otherwise 70 . The real change compared to 2010 is that validation is now commonly referred to as a process including all stages from identification to certification.

3.2.7. In 26 Member States people can take advantage of information and guidance related to validation 71 and in 15 countries guidance is available to support people throughout all phases of a validation process 72 . In the majority of Member States people are offered information and guidance on the outcomes and benefits of validation and on assessment – with particular reference to continuing vocational training and adult learning 73 . Responses to the public consultation confirmed the perception of a certain improvement in information and guidance since 2012 (89 of 150 responding organisations, about 60%) 74 as well as of guidance during the validation process 75 (93 out of 160 responding organisations). On the other hand, one of the reasons why take-up was smaller than expected may indeed be that relevant guidance provision, while increased, is not very effective in reaching out to potential beneficiaries 76 . Only 4 of the 64 beneficiaries responding to the public consultation found their validation opportunity thanks to a guidance centre 77 . The coordination of validation initiatives with existing guidance arrangements is not always clear 78 .

3.2.8. Provision directed to specific target groups has also increased. In 2018, 24 Member States systematically made skills audits available 79 – an explicit invitation of the Recommendation 80 .

Figure 3.2–5. Skills audits made available in 2010 and 2018

Source: Inventory 2010 and 2018. External study, Table 12.

A good number of countries carry out validation initiatives addressing specific target groups such as low-skilled adults or long term-unemployed 81 . Targeted validation initiatives for migrants and refugees – sometimes as specific projects rather than systematic provision – were available in 23 countries 82 . A few arrangements are open to all migrants and refugees (FI, DE, NL, SE and with some conditions AT and DK), while in eleven Member States, such measures are project-based 83 . and as they address specific categories their visibility is relatively limited: about half organisations responding to the public consultation (81 out of 160) considered that validation arrangements in their country targeted disadvantaged groups (long-term unemployed, migrants, disabled persons) either to a high (18) or to some extent (63) 84 . Besides, the cost, complexity and length of validation procedures, along with fragmented provision and uncertain value, remain major barriers for individuals from disadvantaged groups 85 .

Figure 3.2–6 Targeted initiatives for migrants and refugees

Source: Inventory 2010 and 2018. External study, Table 12.

Several Member States are implementing the Recommendation on Upskilling Pathways in close coordination with the development of arrangements for validation. as a way to provide low skilled adults with credit for the skills they have, as a possible bridge to further learning opportunities. This is often driven by the need to upskill workers who have no vocational qualifications, or low skilled unemployed people who public employment services are seeking to return to the labour market 86 .

3.2.9. In 2018, mandatory professional requirements for validation staff – counsellors, practitioners and assessors involved in validation – were reported in 23 Member States 87 . In many countries competence development opportunities are not specifically targeting validation staff. Dedicated training for validation staff is available in EL, NL, PT and is planned in DK and IT.

3.2.10. In 2018, 18 Member States explicitly report that validation arrangements are developed and implemented through multi-stakeholder cooperation 88 . However, analysis of country reports shows a much more frequent participation of a variety of stakeholders in one or the other phase of development and operation of validation arrangements or in activities supporting validation. For instance, labour market actors such as chambers of commerce, employer organisations and trade unions often contribute to standard definition and skills assessment. Public employment services and youth organisations are frequently involved as providers of relevant information and guidance 89 . Interviews to national experts suggest that cooperation of a variety of stakeholders also appears to work as a major factor influencing the efficient of validation provision, facilitating a balanced distribution of costs and burdens 90 .

3.2.11 91 . As asked by the Recommendation, the EQF advisory group has taken over the role of policy group following the implementation. Validation issues are discussed in meetings of the EQF advisory group, including in particular the update of the European guidelines for validating non-formal and informal learning. Within the EQF advisory group, ten Member States volunteered to present one-off reports on validation, based on an agreed structure 92 . Following the adoption of the Recommendation, the Commission has asked Member States to designate each one member and an alternate to this purpose (or assign this task to the member and alternate already following the EQF Recommendation) and has invited new stakeholders’ group to join the group, namely the European Youth Forum and the European Volunteering Centre. The EQF advisory group started meeting in this configuration in its 21st meeting (26-27 September 2013) and validation has been on the agenda in all 30 meetings since then (until November 2019). An extraordinary meeting dedicated to validation took place on 5 May 2019 in Berlin, adjoining the third Validation Biennale, a relevant event organised by third parties 93 . Within the EQF advisory group, ten Member States volunteered to present one-off reports on validation, based on an agreed structure 94 .

3.2.12. In the context of the EQF advisory group, ten peer learning activities specifically addressing validation issues have been organised: on non-formal qualifications (2013, SE), peer learning activities on learning outcomes and validation (2014, IE), on validation between individual pathways and collective strategies (2016, FR), on validation and refugees (2016, NL), on non-governmental stakeholders and the Recommendation (2017, PT), on funding validation (2017, BE), on non-formal qualifications in NQFs (2018, AT), on skills assessment and validation (2018, DE), on validation and volunteering (2019, BE), on peer review and validation (2019, LT).

3.2.13. In 2015 the Commission and Cedefop, following intensive consultation with national authorities and stakeholders, released a revised version of the European guidelines for validation of non-formal and informal learning 95 , fully aligned with the Recommendation. The European guidelines have proven a major instrument to support national validation developments, confirmed by national experts interviewed and suggested by the impressive number of downloads from Cedefop’s website 96 . Three updates of the European Inventory of validation non-formal and informal learning were produced, in 2014, 2016 and 2018 97 .

3.2.14. The 2015 Joint Report by the Council and the Commission on the ‘Education and Training 2020’ strategic framework mentioned the need to reinforce work on validation, as well as on transparency and comparability of qualifications. It specifically made the link to incoming migration and to the innovative learning methods provided by digital technologies. It didn’t however provide any information on the implementation of the Recommendation, as Member States had been asked to take action by 2018. Reporting has rather been provided to authorities and stakeholders through the Inventory (updates of 2014, 2016, 2018) and other documents, such as the regular updates on developments of national qualifications frameworks published by Cedefop.

3.2.15. The Europass framework, for which the Recommendation suggested to explore further developments, has been radically revised in the framework of the Skills Agenda. A 2016 Commission proposal became in 2018 a Decision of the European Parliament and the Council, establishing a common framework for the provision of better services for skills and qualifications (Europass) 98 . The decision states in its Article 3 that the Europass online platform shall provide available information or links to available information on seven topics, one of which is “opportunities for validation of non-formal and informal learning” and that it will make available web-based tools for documenting and assessing skills.

3.2.16. The European Social Fund (ESF) 99 has been a major source of funding for national validation activities, proving particularly effective in supporting wider developments rather than smaller projects 100 . In particular, ten Member States with validation arrangements at an early stage have used ESF resources 101 . Three of them (CY, EL, PL) explicitly relate these ESF-funded activities to the Recommendation. ESF support was however also used to improve validation systems already established, for instance in PT and BE (French community). In most Member States there has been some validation work supported through the ESF, either in projects explicitly focusing on validation or in wider skills development activities. The Erasmus+ programme (2014–2020) 102 has supported EU level activities (Key Action 3, Support for policy reform), such as the peer learning activities on validation issues mentioned above, and a number of strategic partnerships (Key Action 2, Cooperation for Innovation and Exchange of Good Practices). While validation is a complementary issue in many partnerships with another focus throughout all sub-sectors of education and training, it is the core theme in at least 50 projects. Specific objectives of these partnerships include the recognition of skills acquired through volunteering, the validation of skills of specific groups (migrants, prisoners, Roma), and the validation of skills in specific economic sectors such as tourism, construction, wellness, public administration, and maritime. Since 2016, EU resources also became available through the Structural Reform Support Programme (SRSP) 103 . The SRSP has supported the implementation of a national qualifications framework (BG) and is currently supporting three Member States (BE, ES, NL) to improve their skills validation systems. Other relevant projects include raising the quality of adult education systems (HU, PT) and fostering excellence and labour market relevance in vocational education and training (ES, EL, FR, LU).

3.2.17. Only a few Member States (EL, ES, IT, SE) are able to provide information on whether validation beneficiaries find it easier to enter and move within the labour market 104 . This very limited picture is positive. For instance, almost two thirds of respondents to an Italian survey on beneficiaries reported labour market progress following validation. Validation leading to specific professional accreditation (e.g. in security in EL and personal care in ES) was also reported to result in jobs. 

3.2.18. There is little factual information on how validation beneficiaries have engaged in further learning 105 . Sometimes validation processes are indeed embedded in upskilling initiatives run by public instances (e.g. in BE, IT and PT) or social partners (as in DK, FI, IE, SE). While in a majority of countries validation is a way that people may use specifically to access further learning (cf. 3.2.5), there is almost no data on whether a positive validation experience makes people more motivated to go further on their learning pathway. 

3.2.19. In many Member States the costs of validation are not identified, as funding its public provision is part of funding the education and training system or employment policy measures 106 . This is true for countries with well-established systems such as Denmark, Finland and Sweden, and for many countries where provision and related costs are more decentralised. A few countries (BE, ES, FR, IT, LU, NL, RO) do have specific budget items for validation. Two of them (Belgium and Italy) report developments inspired by the Recommendation 107 . A proper cost-benefit study was carried out in Sweden, finding evidence of long-term benefits. As mentioned above, the ESF has provided the resources for many validation activities, especially the development of initial arrangements. In several countries, use of ESF resources was related to developments in line with the Recommendation, or directly inspired by it (Cyprus, Greece and Poland). Several representatives of national authorities have reported that validation benefits exceed its costs, sometimes referring to specific analyses 108 . In some countries, distribution of costs over the actors seems unbalanced. The main factors influencing the efficiency of validation have been identified in multi-stakeholder cooperation and effective targeting 109 . A partnership of many actors allows a more balanced distribution of costs and a fairer allocation of burdens. The improved dialogue triggered by the Recommendation has for instance helped mainstreaming resources in Finland and debate sustainability in Sweden 110 . Inappropriate targeting of measures addressing specific groups has resulted in fragmented management and short-term planning, with limited results.

4.Method

Short description of methodology

This evaluation was organised following the Better Regulation Guidelines, addressing the five criteria of effectiveness, efficiency, coherence, relevance and EU added value.

An external study supporting the evaluation was carried out. The study included desk research, field research, including a public consultation, and the development of conclusions and lessons learnt. The methodology is described in Annex 3.

Limitations and robustness of findings

The European Inventory on validation and its updates are the main source of information on the situation of validation in Member States 111 . Country reports are prepared by external consultants and proofread by representatives of relevant national authorities (since the 2014 update this is done by members of the EQF advisory group). While those consultants retain their responsibility for the deliverables sent to the Commission, namely for their analytical dimension, national representatives check the accuracy and may clarify misunderstandings, e.g. in relation to the legal framework for validation, ensuring a good quality of the information on national validation activities.

From one update to the next, information collection has become more structured and has gone into more detail, also with reference to the Recommendation. While this may require some adjustments, it remains to a large extent possible to compare over the years information on the main categories, such as the availability of validation arrangements, the provision of guidance, the link to qualifications frameworks and to institutional education and training. The Inventory update of 2010 and to some extent the update of 2014 do represent a solid baseline for the evaluation of a Recommendation adopted in 2012. The 2018 update was conceived with the explicit purpose of gathering a robust knowledge base supporting the evaluation of the Recommendation.

Further reliable information sources are also available. The one-off reports that some Member States have presented to the EQF advisory group are accurate and up to date. The study on skills audits published early in 2019 provides a detailed picture of the situation in all Member States in 2018. Other useful literature is also available on relevant EU initiatives and national developments, such us the regular reports on national qualifications framework prepared by Cedefop.

Input from interviews, expert meetings and the public consultation have allowed to complete and clarify the information available in the Inventory and other literature, supporting a cross-reference analysis. Input has sometimes complemented limited information, e.g. representatives of authorities in several countries have declared that in their country the benefits of validation exceed its costs – an issue on which secondary data are scarce. In other cases, input has corroborated findings from desk research; for instance, results from the public consultation show that some validation opportunities are available in most Member States, but their provision is often limited and typically not available to everybody, thus confirming the picture provided by the 2018 update of the Inventory.

While the overview of validation in Member States provided by the European Inventory is a good information source on most relevant issues, there are however some objective limitations as concerns the availability and comparability of some categories of information. In spite of progress, data on the take-up of validation opportunities and the costs of validation remain limited. This makes it difficult to fully assess the effectiveness of the Recommendation, in terms of increased number of people benefitting from validation opportunities, and its efficiency, in terms of proportionality of costs compared to the benefits brought to individuals, organisations and society.

While in the 2018 update of the Inventory most Member States reported some information on the take-up of validation arrangements, data are not comprehensive. They are typically collected on some but not all subsectors in which validation arrangements are in place. For instance, they are collected for candidates to lower level vocational training qualifications, but not for people seeking exemption from parts of programmes in higher education or higher vocational training 112 . Besides, as data were mostly not available before the adoption of the Recommendation, comparison is not possible or not reliable. Information about increase or decrease in take-up by potential beneficiaries is contained in the country reports of the 2018 update of the Inventory, but it either refers to comparison 2016 or to an assessment based on partial and local data. However, available data and triangulation with interviews and the public consultation make it possible to formulate well-grounded assessments.

Clear information on the costs of validation is limited as such costs are mostly part of wider cost items related to education or employment policy. While public funding is the most common resource for validation, it often is associated with other sources, from individuals that pay a fee to companies and private organisations to specific projects 113 . Measuring the cost of validation is objectively difficult as it is a process where several actors may be involved, such as dedicated validation services, information and guidance centres, education and training institutions, Chambers of commerce and non-governmental organisations 114 .

These limitations contribute to make it difficult to assess the direct impact of the Recommendation on the overall positive evolution 115 . Relevant national documents rarely refer to the Recommendation. In many cases national efforts that implement the provision of the Recommendation may also be seen as the continuation of own national strategies and initiatives. However, interviews with representatives of national authorities did result in some explicit statements on the impact of the Recommendation. The public consultation has also contributed to gather views on the role of the Recommendation in increasing validation opportunities. A reasonably valid assessment of the impact of the Recommendation is therefore possible.

5.Analysis and answers to the evaluation questions

5.1 Effectiveness

The information reported in section 3 shows that there has been some progress in making it easier for people to access validation opportunities (evaluation question 1.1.a) and in making it possible to use validation results to learn and study across Europe (evaluation question 1.1.b). It also shows that there still is ample room for further progress, namely in making validation opportunities available to any individual at any stage of life, along with appropriate support. Taking into account the non-binding nature of the Recommendation, the action taken by Member States shows a good level of effectiveness, though more and better action is needed to achieve the objectives of the Recommendation.

1.1.a – People have more validation opportunities than in 2012 because Member States have responded to the first invitation of the Recommendation and set up or improved validation arrangements [3.2.1 116 ], taking advantage of the principles suggested in invitation 1.3 [3.2.3 to 3.2.9] and widening stakeholders’ cooperation [3.2.10], as called for by invitations 1.4 and 1.5. However, the relative fragmentation of provision, restricted access, and lack of support still prevent many people in taking advantage of validation opportunities [3.2.2].

Figure 5.1.1 Validation arrangements in place in 2010 and 2018

Source: Inventory 2010 and 2018. External study, Figure 3.

While 132 of 160 organisations responding to the public consultation (83%) considered that validation opportunities were available in their country, only 54 of 65 EU individual respondents shared this view and 30 individuals believed they were not available 117 . This suggests that fragmented provision of opportunities makes them less visible to individuals and confirms that information and guidance do not sufficiently reach out to potential users [cf. 3.2.7].

Besides, many respondents added a free text comment underlining the limitations of validation arrangements, with access to validation being limited to certain categories or made difficult by cumbersome procedures 118 . The external study found that most validation arrangements are not comprehensive, all envisage some limitation to access and some are still quite limited in their scope [3.2.1]. On the one hand, this is in line with the pragmatic approach of the Recommendation, which in its invitation 1.1 said that Member States could “prioritise certain areas and/or sectors”. On the other hand, it should be clear that action by 2018 was a first step towards validation available to everybody, which remains the objective and is far from being achieved. Some interviews and discussions in the expert group showed awareness that the Recommendation was “only the beginning of a longer-term approach to validation”  119 . Most respondents to the public consultation (198 out of 257) agreed that people should be able to validate their skills in all cases, while 55 thought this should happen in particular cases 120 . Progress has been real, but limited, also with reference to disadvantaged groups. In spite of an increase in specific opportunities targeting them, in practice for many people from such groups access remains difficult [3.2.8].

As reported in Section 3.2.2, more people have used a validation service, though not as many as could be expected, and in a few countries, use of validation has actually slightly decreased. Individuals who might benefit from validation may not use it because they lack the necessary support. This is confirmed by responses to the public consultation: 23 of 62 beneficiaries of validation were satisfied with the guidance during the validation process, 17 were not satisfied and 15 had not had any 121 , while financial support had only been available to seven of 62 responding beneficiaries 122 [3.2.2, Figure 3.2-3].

The increase in opportunities and, however limited, in their use has also taken advantage of the now widely shared understanding that validation is a process including different stages [3.2.6], each of which has its own value and should be separately available to people (invitation 1.2). This seems to be a frequent case, as for instance validation initiatives in the third sector [3.2.1], skills audits for the unemployed, and initiatives targeting specific groups [3.2.8] do not go beyond identification and documentation. The public consultation confirms this: on the one hand most responding organisations were well aware of the four stages, with more than 100 out of 161 considering each phase was reasonably accessible in their country 123 ; on the other hand only 14 of 64 responding beneficiaries had gone through the whole process and received a qualification, while others didn’t go beyond identification (15), documentation (7) or some form of assessment (11) 124 .

More relevant actors are playing a role in validation [3.2.10] 125 , and this may have contributed to make it easier for some people to access validation. Out of 160 organisations responding to the public consultation, 114 considered that all relevant parties were involved in national validation developments to a high (51) or some extent (63) 126 . On the other hand, the fact that most validation arrangements are not comprehensive and many people do not have access to validation or do not take advantage of it [3.2.2], suggests that more strategic coordination of actors, each of which may already be playing an active role in validation, would more effectively pursue the objectives of the Recommendation.

1.1.b – More people find it easier now than in 2012 to use their validation results across countries, namely when they obtain a full qualification through validation. In fact validation can now lead to qualifications included in the qualifications framework (invitation 1.1.b) in most Member States – though in only about half Member States validation can lead to any qualification in the framework [3.2.3]. This is facilitated by widespread equivalence of standards between validation and formal education and training, called for by invitation 1.3.h [3.2.4]. The figure below shows that, thanks to the close link between validation arrangements and national qualifications frameworks referenced to the EQF, implementing invitation 1.3.a, in 2018 in most countries people who obtain a qualification through validation will be able to use it to learn or work across Europe.

Figure 5.1–5 Validation and national qualifications frameworks in 2018

Source: External study, Figure 10. Inventory 2018. Synthesis Report, p.27.

National qualifications frameworks are increasingly including qualifications awarded outside institutional education and training, sometimes called “non-formal” or “non-traditional” qualifications 127 , such as the more and more common micro-credentials 128 . Recent research has highlighted the need for formal education and training, credit systems and qualifications frameworks to take into account micro-credentials and related developments 129 , which could have a disruptive impact 130 , but could also “bring together the world of education and training and the world of work” 131 and boost the flexibility of learning pathways 132 . A more coordinated development and implementation of validation and qualifications frameworks might generalise validation as a pathway to all qualifications.

1.2 – The contribution of the Recommendation to the (partial) progress at national level seen above took several forms, as learned from interviews with national experts 133 . In systems at an earlier stage of development, the Recommendation has helped shaping them (IT, MT). It has also supported existing systems by adding weight to national action (BE, ES), making the approach more comprehensive (SI), giving a new strategic direction (FI, SE) or making validation more visible at national level (CZ). It also influenced initiatives targeting specific groups (BE, IT, PT, SI) 134 .

The information reported in section 3 [3.2.1 to 3.16] shows that the follow-up and support actions called for in invitations 2 and 3 have been implemented. The only exception concerns invitation 2.b on using the Education and Training 2020 Joint Report, cf. next paragraph.

2.1 – The follow-up activities envisaged by the second invitation of the Recommendation have been mostly implemented. The EQF advisory group had become the validation policy group [3.2.11] and Cedefop has provided a major contribution to the follow-up, including through its reporting activity 135 . The 2015 Joint Report by the Council and the Commission on the ‘Education and Training 2020’ strategic framework, prepared less than three years after the adoption of the Recommendation, has not proven to be an effective reporting tool for validation[3.2.14].

2.2 – Some contribution to the above mentioned progress may have come from the support activities envisaged in invitation 3, which have all been implemented as seen in sections [3.2.12, 13, 15, 16], mostly covered by the Erasmus+ budget. National efforts to widen validation opportunities have taken advantage in particular of the European guidelines [3.2.13] 136 . Some suggestions for improvements were collected. For instance, peer learning activities could become more effective through closer coordination with similar activities related to relevant initiatives and with a higher profile 137 .

At this stage, the limited information available does not allow to draw any well-grounded conclusion on the impact of validation on its beneficiaries: whether validation helps them finding a job or a better job, and whether they engage in further learning. Only a couple of countries carried out targeted surveys on validation beneficiaries, which suggested a positive impact [3.2.17 and 3.2.18], but they cannot support an answer to this question at European level. As data on uptake in general remains rather limited, a systematic follow up of validation beneficiaries is at this stage not possible.

5.2 Efficiency

There is not enough information on costs and benefits directly related to the Recommendation (or indeed to validation activities themselves) to be able to compare them or to assess the proportionality of costs. Interviews and the public consultation express positive views, but don’t represent solid evidence. Stakeholder cooperation is recognised as an efficiency factor.

Considerations of the cost-effectiveness of the Recommendation depend on a very thin information basis (cf. section 4.2). It is very difficult to identify the cost of validation as separate from costs of wider education or labour action, and detecting costs determined by the Recommendation, and therefore reviewing their proportionality, is not feasible. As there is a certain consensus that some national action might not have been taken without the Recommendation 138 , this suggests that costs may have been higher. However, any such increase in cost cannot be identified, as the stimulus to action due to the Recommendation typically had an impact on existing systems and ongoing initiatives. Besides, only a few countries have dedicated funding frameworks for validation, which is often covered through wider budgetary items such as education or employment 139 . In several countries early-stage arrangements have developed into better established systems, often supported by ESF, which in the long run should result in better cost-effectiveness 140 . Information on benefits is also very limited, as seen above [3.2.17 and 3.2.18]. The qualitative judgement of national representatives, as gathered in the external study through interviews, is that validation benefits exceed its costs [3.2.19], although this refers to validation rather than to the Recommendation. A positive view on proportionality of cost, with reference to the Recommendation, was expressed by more than half organisations responding to the public consultation (86 out of 160); however most others (57, more than one third) did not feel able to reply, confirming the lack of solid information 141 .

When it comes to factors influencing efficiency, a positive result is that the Recommendation “has been reported to contribute to rationalising the use of financial and other resources in the development and provision of validation” 142 , therefore increasing the efficiency of validation activities in Member States. To a certain extent, this happened because the Recommendation inspired comprehensive stakeholder partnerships, which appear to be a significant efficiency factor [3.2.10].

5.3 Relevance

There is wide consensus among national authorities and stakeholders that the Recommendation remains relevant in today’s social and economic conditions 143 . Specific measures may need adjustments, wider representation of stakeholders could make governance more relevant, and it would be helpful to include provisions for a solid evidence base.

5.1 – The objectives and principles of the Recommendation provide a useful contribution to addressing the rapidly evolving skills challenges 144 . In several Member States the Recommendation has promoted national action and its potential has been highlighted with particular reference to the skills needs related to digitalisation and innovation of work practices 145 .

The Recommendation is seen as responding to the needs of the different stakeholders, although its high-level provisions may more easily support policy stakeholders than practitioners. Two thirds of the organisations responding to the public consultation (103 out of 156) agreed that it appropriately addressed their needs, though one in five (32) was not able to reply 146 .

The call for involvement of all relevant stakeholders was often highlighted and considered to deserve stronger emphasis at EU level 147 . More guidance on how to put in place wide ranging stakeholders’ partnerships, building on the 2015 European guidelines for validating non-formal and informal learning, would be beneficial 148 .

5.2 – Specific measures and principles are also mostly judged as still relevant. The flexible approach to the four stages of the validation process fits with the need for both national adaptations and individual pathways, although further guidelines might prove helpful 149 . The link to national qualifications frameworks retains its importance, though developments could better take into account the increasingly rapid evolution of skills and occupational standards 150 .

Interviewees from several Member States have noted have noted the relevance of the Recommendation with reference to the inclusion of disadvantaged groups, including migrants 151 . There was, however, some concern that so far disadvantaged groups had not been among the main beneficiaries of the Recommendation, probably because of inadequate outreach. It is possible to achieve higher relevance to the needs of disadvantaged groups through close coordination with interventions from other policy fields 152 .

While the four stages approach has been an effective factor promoting a shared understanding of validation throughout Europe [3.2.6], the meaning of the assessment and the certification phases might need some reflection to remain universally relevant in a changing context 153 . Emerging practices in skills recognition, such as micro-credentials and other digital methods, may require a re-definition of the certification phase. The equivalence of standards, traditionally focused on formal education and training and their qualifications, may also need some reflection (cf. above, section 5.1).

5.3 – The relevance of follow-up and support measures, while acknowledged, could improve, as suggested by some interviewees and participants in the expert meetings, by involving a wider range of stakeholders (for instance through validation events) and ensuring a solid information basis that would convert the Inventory into an instrument for actual monitoring as is usually carried out in the framework of the open method of coordination 154 .

5.4 Coherence

The Recommendation is coherent with European policies and tools in related fields. Some of its provisions, while conceptually coherent, may be less coherent with specific tools implemented nationally. The Recommendation is coherent with national validation policies, although some of its provisions are not fully implemented.

Most national experts interviewed recognise the conceptual and thematic coherence of the Recommendation with other EU policy areas and instruments 155 . The principle of learning outcomes is the red thread that ensures consistency across relevant EU policies and tools. While there is clear coherence with major tools such as the EQF and recent initiatives such as Upskilling Pathways 156 , some national experts noted an insufficient practical and organisational coherence with specific policy tools. In particular, the synergies between validation and credit systems such as ECTS and ECVET, specifically called for in the Recommendation, do not seem to have been achieved beyond a few cases 157 . While the 2018 Inventory reports the in most Member States validation can result in credit [3.2.5], the external study notes that there it is unclear how this relates to the use of ECTS and ECVET 158 . Most Member States implement ECVET as an instrument to support flexible vocational pathways and rarely as a credit point system 159 , which reduces the coherence of the Recommendation with this tool 160 . On the other hand, use of ECTS is standard practice in higher education throughout Europe, but link to validation is equally not clear in many countries. The links between validation and recognition of small sets of skills is an issue that deserves further development, also in relation to innovative practices of skills recognition such as micro-credentials, which are often linked to short courses, but may also have an impact on validation activities 161 .

There is wide consensus among interviewees from Member States that the measures of the Recommendation are consistent both with each other and with national validation policies 162 . This is particularly apparent in countries that adapted their relevant legal framework or strategies 163 . General consistency with national policies coexists with different levels of practical implementation of the principles of the Recommendation. For instance, as seen above [3.2.1] access to validation, while not comprehensive in any country, is more or less wide depending on the scope of validation arrangements. Stakeholders who noted some lack of coherence referred indeed to the gap between the limited access made available by national action and the general access promoted by the Recommendation 164 .

5.5 EU added value

There is a certain consensus that in several countries the Recommendation has had a positive impact on national action towards its objectives. Appropriate public discourse would support the more common understanding of validation that is emerging throughout national policies and arrangements. Experts from about half Member States considered that further EU action in the field is necessary or appropriate, and could gain if it were more specific and closer to the operational level.

Interviews with national experts suggest that the Recommendation has had a certain positive impact on national action in a number of Member States, to different extents and in different forms 165 .

In countries where validation arrangements in 2012 were less developed, the Recommendation has been beneficial contributing to the policy debate (e.g. inter-ministerial discussions on validation in Slovakia 166 ) and to actual development (e.g. supporting the legislative process in Italy 167 ).

Representatives from countries with better established arrangements also recognise that “the CR has given some impetus to already existing national validation strategies or actions”, or that the Recommendation has had a positive impact in shaping strategic choices 168 .

A huge majority of organisations responding to the public consultation (100 out of 160) agreed that the Recommendation had played a role in national action towards more and better validation opportunities and slightly less (91 out of 160) that it had enabled individuals to progress on their learning pathway or career 169 . Free-text comments highlighted the role of the Recommendation in fostering the debate on validation and stimulating developments.

External study, Figure A.3.14.

A slightly smaller proportion of responding organisations (95 out of 160 or 60%) considered that the Recommendation had indeed contributed to making validation more available to people 170 . 

National arrangements are largely conceived and practised “in accordance with national circumstances and specificities” (invitation 1 of the Recommendation), and the Recommendation does not aim at imposing one specific validation approach to the diverse national policies. However, a number of interviewed national officials and participants in expert meetings recognise that its principles have promoted a more common understanding of validation across Europe 171 . The widespread view of validation as a four stage process mentioned above is a clear indication of this positive trend towards a shared understanding, which will facilitate people to use their validation results across countries. Documentary sources suggest however that in a few countries the concept of validation needs further clarification 172 . Free-text comments of organisations responding to the public consultation note that more public discourse on validation would facilitate a common vision within and across countries 173 . 

During interviews, 11 key informants also reported a certain impact on national policies in areas other than validation. Which specific areas varies between countries, with seven however agreeing on some impact on career guidance 174 .

Key informants from 11 countries, along with EU level key informants, considered continued EU action necessary to achieve the objectives agreed 175 , while for five others EU action could play a role in promoting comprehensive national systems 176 . Several experts interviewed noted that EU action could better support national action if provisions were less generic, for instance better defining target groups or more closely linking to the operational European guidelines 177 .

6.Conclusions and lessons learned

6.1 Conclusions

The ambitious objectives of the Recommendation have not been fully achieved, but there is evidence of significant – albeit fragmented – progress since 2012. All Member States have taken some relevant action by 2018, using to a large extent the principles suggested, although this has not yet resulted in systems open to everybody. The follow up and support activities carried out were appreciated by stakeholders, who nevertheless identified the need for further improvement and made suggestions for how to achieve this. The Recommendation called for stakeholders’ coordination and in cases where this has been put into practice, it has indeed proven to be a factor making national action more efficient. The objectives and measures of the Recommendation are considered still fully relevant in the current socio-economic situation. Their general coherence with the policy context is also acknowledged, although coordination with policy tools at operational level could improve. The EU added value is most visible in the higher policy profile and increased practice of validation observed to different degrees in all Member States, in some of which national action alone would not have reached the same extent. The evaluation has identified areas where further action is needed to achieve the goals of the Recommendation, to provide people with access to more and better validation opportunities, enabling them to access further learning and to put their skills to good use in European society and the labour market.

Achievements

Most countries’ arrangements are still far from being comprehensive, but there has been progress towards providing more validation opportunities 178 . Some degree of validation arrangements have been set up or strengthened in almost all Member States, including by actors in the labour market and the third sector. To some extent, validation has become more accessible, and based on the limited available data, it appears that take-up by people has increased to some extent 179 . The Recommendation is recognised to have contributed to this progress, by raising the profile of validation in the policy debate and in some countries stimulating action that might not have been taken otherwise 180 .

The four stages of validation are acknowledged to be a sound common reference framework, applied with terminological variants 181 , and Member States have to a large extent adopted the principles suggested in the Recommendation when developing validation arrangements 182 : there are clear links to national qualifications frameworks, now available across Europe, there is more information and guidance, many initiatives target disadvantaged groups, quality assurance has improved and equivalence of standards with formal education is common, transparency tools and formal credit systems are used to some extent in connection to validation.

Challenges

In spite of progress, most validation arrangements have limitations to access 183 . In many countries, young people with no work experience cannot apply for validation that leads to full or partial qualifications. Validation can help them access formal education, but not the labour market. The Recommendation does not call for centralised, unified national validation systems, but does call for arrangements that “enable individuals” (with no qualifier) to have their skills validated. In action taken by 2018, Member States have prioritised certain sectors or areas. This was a realistic approach in 2012, as such pragmatically envisaged by the Recommendation, but it inevitably excludes some categories of potential users.

While data remain limited, they suggest that fewer people than expected have engaged in validation, despite larger provision of opportunities and guidance 184 . There may be several reasons. Of course, limiting access as mentioned above reduces the number of eligible individuals. Even for them, in practice access can be difficult because validation opportunities are provided by a variety of agencies with little coordination. Guidance, while available, may not have been effective in reaching out to potential beneficiaries 185 . Finally, if there is no active support to individuals, such as paid leave or a financial contribution, they may not be able to engage in validation 186 .

Reaching specific disadvantaged groups, such as long-term unemployed or migrants, remains a challenge, despite being the focus of an increasing number of initiatives 187 . While data on their outcomes are limited, impact is often reduced by ineffective reach out, procedural complexity and uncoordinated provision on the one hand, and on the other hand, the high level of individual commitment required. This often results in inefficiencies, with potentially good initiatives not fully exploited.

Scarcity of information is an issue through all aspects of validation provision 188 . While there is good information on the structure of validation arrangements and the related policy and legal frameworks, there is little and fragmented information on their operation, costs, outcomes and impact. There are occasional surveys on beneficiaries and organisations involved, but no systematic tracking. The information available is not enough for any proper monitoring, and benchmarking could only be based on a few qualitative indicators with limited value, such as the existence or not of a specific legal framework.

6.2 Lessons learned

Providing more validation opportunities is not enough. Providing support to individuals is necessary 189 . A core principle of the Recommendation, and of European skills policy, is that everybody should have opportunities to validate their skills 190 . In 2012 it was pragmatic to focus on providing “more opportunities”; in 2020 it is necessary to open up opportunities and support individuals to actually take advantage of them 191 . People who are aware of the benefits of validation and of available opportunities may nevertheless wonder whether the benefits are worth the costs. They need to be motivated enough to invest time, energy and often money over a sustained period 192 , all at the expense of work and family. To increase their motivation, investment should be shared with employers (e.g. paid leave, technical assistance), authorities (e.g. financial support, coaching, simplified procedures), and other actors such as unions, trade organisations, and education and training providers 193 . Specific support should address the barriers limiting the access of individuals from disadvantaged groups 194 .

Stakeholders’ cooperation is not enough. There is a need for strategic coordination. While a variety of agencies providing validation can be a richness, responding to the needs of a diverse set of beneficiaries, a closer coordination of providers and cooperation with all stakeholders could improve both effectiveness – through better visibility, wider reach out, operational synergies – and efficiency, by distributing burdens, sharing facilities and peer learning 195 . The Recommendation does stress stakeholders’ involvement and coordination (points 4 and 5). However, while many individual validation initiatives reach a good level of stakeholders’ cooperation, coordination is weaker between validation initiatives and between validation and other areas of skills policy 196 .

Providing more guidance is not enough. Closer cooperation and effective coordination between providers of guidance and validation is needed 197 . While relevant guidance should be “made available to all individuals and organisations”, as the Recommendation says 198 , in any time and place there is a need to tailor the media and the message to the audience: to reach out to them, to convey the right meaning. For instance, to illustrate to vulnerable workers the benefits of validation with a view to upskilling, and guiding them to the most suitable validation pathway. This would promote take-up in general, and in particular improve effective tailoring of validation initiatives addressing disadvantaged and vulnerable groups, making them more effective and efficient 199 , in coordination with the implementation of the Recommendation on Upskilling Pathways 200 .

A formal link to national qualifications frameworks is not enough. Coordinated implementation of validation and national qualifications frameworks is needed 201 . The emergence of qualifications frameworks and their alignment to the EQF, with their central concept of learning outcomes independent from learning settings and inputs, has been a major factor in promoting validation action in most Member States. While well-established in the Recommendation, and effectively pursued through the EQF advisory group, the link between validation and qualifications frameworks needs to develop in its practical implementation 202 .

A link to formal credit and qualifications is not enough. Validation should take advantage of innovative practices in skills recognition, such as micro-credentials. As micro-credentials recognise a small or very small set of skills, they can be a useful instrument to validate skills developed outside formal education and training. If they are developed and awarded respecting agreed standards of quality assurance and transparency, micro-credentials could make learning pathways more flexible 203 . Appropriate developments, closely coordinated with qualifications frameworks, could build upon the experience gained through the implementation of ECVET 204 and national experience with partial qualifications 205 .

The current level of information is not enough. Systematic information collection on validation is necessary. Validation has grown in all Member States in terms of policy significance and operational developments. Still, the poor and fragmented information available makes it difficult to meaningfully assess the relevance of policy measures, the effectiveness of their implementation, and the proportionality of investments. The Recommendation calls for updating the European Inventory of validation, which is recognised as a valuable tool 206 , but does not provide for any systematic information collection at national level that could support the quality, accuracy and relevance of the Inventory. With a proper information basis, it would be possible to agree on indicators and monitor progress 207 .

Annex 1: Procedural information

1.Lead DG, Decide Planning/CWP references

The evaluation of the Recommendation was led by the Directorate-General for Employment, Inclusion and Social Affairs.

2.Organisation and timing

An evaluation roadmap published on 16 October 2018 explained the context, purpose and scope of the evaluation and informed stakeholders that an external evaluation study would be carried out, building upon the 2018 update of the European Inventory of validation and information provided by national authorities within the EQF advisory group, and that a public consultation would be launched.

An Inter-service Steering Group (ISSG) was set up, coordinated by DG Directorate-General for Employment, Inclusion and Social Affairs (represented by staff from the lead policy unit, another policy unit and the evaluation unit) and including members from the Secretariat General, Education and Culture, the Joint Research Centre. The ISSG could take advantage of the technical assistance of expert staff from Cedefop and the European Training Foundation (ETF).

The ISSG also acted as steering group for the external study supporting the evaluation, which was carried out by ICF Consulting Services. Contract VC/2019/120 was awarded after reopening of competition (tender VT/2018/035) within the multiple framework contract “Provision of services related to the implementation of Better Regulation Guidelines” (VT/2016/027). The contract VC/2019/120 started on 1st April 2019 and lasted eight months, until 31 December 2019.

The draft final report was received on 4 December 2019, the inter-service steering group provide comments by 20 December 2019 and the contractor delivered the final report on 20 January 2020, along with the agreed annexes on the stakeholders’ consultation

The external support study had been announced in the evaluation roadmap published on 16 October 2018. It included a public consultation, which remained open 13 weeks (7 August 2019 to 13 November 2019).

The preparation of the evaluation was discussed with the EQF advisory group in its meetings of 2nd October 2018, and interim results were presented in the meeting of 4-5 November 2019.    

3.Exceptions to the better regulation guidelines

All Better Regulation requirements were fulfilled.

4.Consultation of the RSB (if applicable)

Not applicable.

5.Evidence, sources and quality

A specific study supporting the evaluation of the Recommendation was carried out in 2019. The final report is available here . It included a public consultation (7 August to 13 November 2019), which resulted in 261 responses. The contractor also organised two expert workshops, focusing on validation in the labour market and on validation and qualifications frameworks.

The main sources of information were:

–the European inventory of validation non-formal and informal learning 208 . The 2010 update, and to a lesser extent the 2014 update, provided the baseline information, while the 2018 update, finalised in summer 2019, provided information updated to the end of 2018;

–interviews with 72 experts from national government bodies, qualifications authorities, learning providers, validation practitioners, and stakeholders’ associations;

–literature from research and academia.

Complementary information included a series of reports that national representatives of 6 countries (Germany, Latvia, Portugal, Luxembourg, Denmark, UK) have presented to the EQF Advisory Group over 2019 (one-off reports on validation) 209 .

The present Staff Working Document relies upon the final report of the specific study as well as on direct consultation of the European inventory of validation of non-formal and informal learning, the national one-off reports on validation. Information acquired in peer learning activities of the EQF advisory group as well as through the ordinary coordination of the EQF advisory group. Available literature was equally consulted, in particular publications from the OECD.

Annex 2: Stakeholder consultation

Synopsis Report on stakeholder consultation activities

In accordance with the Better Regulation Guidelines, this Annex provides a synopsis or summary of all the consultation activities conducted within the evaluation.

The stakeholder consultation activities were carried out in the framework of the external study supporting the evaluation 210 :

·Key Informant Interviews (KIIs)

·Expert group meetings

·Public consultation

1 - Key informant interviews (KIIs)

In total, 72 KIIs were conducted. In some Member States, no KIIs could be conducted (DK, LV) while in several others fewer than two were completed (EE, LT, RO, UK). This was mostly due to lack of responsiveness or lack of detailed knowledge of the Recommendation among some of the targeted stakeholders.

Table A2.1. Number of KIIs completed by Member State and at EU level

The following table shows the type of stakeholders taking part in the KIIs for this evaluation study.

Table A2.2 Overview of KIIs completed by stakeholder type

The topic guide used to collect information from the KIIs included 25 questions on the five evaluation criteria, effectiveness (14), efficiency (3), relevance (3), coherence (2), EU added value (3).

2 - Expert group meetings

Two Expert group meetings were held in Brussels to enable a reflection on the evaluation’s interim findings and to facilitate the exchange of experiences and observations among stakeholders the implementation of the CR (in different Member States and the EU) and on the topic of validation more generally.

The two thematic meetings held were as follows:

A total of 15 participants included three from European umbrella organisations (the European Youth Forum and European Association for the Education of Adults) and 12 from ten countries (BE-Fr, CZ, ES, IE, IT, MT, PL, SE, SK and Norway), specifically from national ministries (2), national agencies for education and training (2), regional-level labour organisations (2), training and academic institutions (2), an organisation in charge of validation (3), and one from a business organisation.

Below is a summary of the key messages from the Expert group meetings.

The role of employers and other labour market actors in validation arrangements

·The CR is effective and relevant in helping Member States conceptualise validation and in creating momentum for multi-stakeholder collaboration involving employers and labour market actors.

·The Recommendation has generated considerable added value in those Member States where validation is still in its early stages of development.

·Limited evidence overall as to whether the Recommendation has enabled people to use their validation outcomes for entering the labour market and progressing within it.

How validation arrangements relate to national qualifications frameworks (NQFs) and the shift to learning outcomes and flexible learning pathways

·The Recommendation has helped establish learning outcomes (knowledge/skills/competences) as a ‘currency’ for validation across the EU.

·The intended objectives of the Recommendation are restrained by the fragmentation of validation processes observed in many countries, compounded by the lack of a common vision among different stakeholder types in some cases.

·While the Recommendation has helped shape a more common understanding of validation across the EU, there is very limited evidence of validation outcomes being used for intra-EU mobility purposes.

Overall conclusions

·The Recommendation is regarded as having continuously fed into discussions on validation in many Member States and has been used as for developing validation processes in those countries where they were mostly inexistent prior to 2012.

·Validation cannot operate separately and needs to be embedded in wider skills and lifelong learning strategies, which requires institutional change to facilitate multi-stakeholder collaboration.

·It may still be too early to assess the extent to which the Recommendation and other relevant EU-level instruments can be applied to improve links between validation and formal education systems, to develop flexible learning options as a gateway to validation and to facilitate EU mobility.

·Validation does have a bright future as it can be expected that skills will increasingly require updating to remain relevant to the rapid evolutions of the labour market.

3 - Public consultation

The public consultation has served to gather views on the Council Recommendation from the wider community of experts and practitioners on validation on the one hand, and from people who have undergone a validation process. Certain questions were therefore only targeted at organisations acquainted with validation while some others were specifically targeted at individual end-users of validation.

Following its translation into 22 other European languages, the public consultation was launched on 7 August 2019 and closed on 13 November 2019. It was disseminated to the relevant networks of DG EMPL with the targeted networks having been requested to disseminate the public consultation to their respective beneficiaries and partners.

National ministry stakeholders taking part in the KIIs have also been asked to disseminate the public consultation to their relevant networks, while national-level validation and career guidance organisations have been encouraged to do likewise with their beneficiaries.

The public consultation generated a total of 262 responses. In addition, ten organisations submitted a position paper together with their public consultation responses: seven at the national-level and three at the EU-level.

Due to a few incomplete or ‘blank’ responses, sample sizes show small variation across the questions. To that end, sample sizes are provided below each chart for reference.

Responses came from all Member States, plus Turkey (5), Albania, Armenia, Morocco North Macedonia, Norway, Pakistan and Switzerland (one each). Respondents from Italy were most numerous with 27 responses (10%), followed by 23 responses (9%) from Portugal and 18 (7%) from the United Kingdom. Cf. Figure A2/1 below.

Figure A2.1 Overview of responses by country (External study, Figure A3.1)

N=262, Source: public consultation results

In terms of participant type, the 262 responses split unevenly between organisations (163 responses or 62%) and EU/non-EU citizens (99 responses or 38%). This sample size provides a good basis for analysis as both groups are sufficiently represented, whilst it also allows for further segmentation by type of organisation.

When analysing the type of organisations in the sample, public authorities are most prevalent (44 responses), followed by NGOs (38 responses) and academic/research institutions (31 responses).

Table A2.3 Type of organisations in the sample (External study, Table A3.1)

N=163, Source: public consultation results

Questions for all respondents

The first three public consultation questions were addressed to all respondents.

Q1 - To your knowledge, are there possibilities for people in your country to validate the skills they have acquired outside school or university?

Nearly three out of four respondents (72%) thought this was the case, 20% believed that such possibilities were not available, whilst 7% didn’t know. Rather unsurprisingly, a significantly larger share of representatives of organisations were aware of validation opportunities compared to citizens (28pp difference).

Figure A2.2. Q1 - To your knowledge, are there possibilities for people in your country to validate the skills they have acquired outside school or university? 
(External study, Figure A3.3)

Source: public consultation results

Well-established systems are well known: almost all respondents from France and Sweden believed that validation opportunities were available.

A total of 88 open-text answers were received to further clarify and explain responses to Q1. Whilst the views are rather heterogeneous, most responses refer to the limited availability of validation arrangements. These are often provided through specific projects and/ or in relation to VET and hard skills. Put differently, well-developed and nationwide procedures are still seldom in place, thus further efforts are required according to the responses.

Q2 - Do you think that people who acquired skills in the workplace or outside school should be able to have them validated?

Nearly all respondents agreed, with 4 out of 5 selecting ‘Yes, in all cases’ and the others ‘Yes, but only in particular cases’. Two non-governmental organisations disagreed.

Q3 - Have you personally participated in a programme to validate skills you acquired (through work, community groups, volunteering etc.) outside an education programme?

One out of four respondents have participated in a programme to validate skills acquired outside an education programme as shown in the figure below. They were therefore invited to respond to questions .

Questions for organisations

Questions 4 to 13 were addressed to organisations.

Q4 To what extent can people in your country obtain /identification/ documentation/ assessment/ certification/ of their skill?

Some 60% of respondents believed in relation to all four stages that these can be obtained to a high extent or to some extent. A small minority (max 7%) believed there was no opportunity.

A total of 34 respondents provided further comments. Several comments describe hindering effects, including complex and lengthy validation processes, the limited possibilities for identification and documentation of skills, as well as limited awareness about the validation possibilities. As pointed out above, validation often covers only a set of professions and skills rather than being offered on a universal basis.

Q5 Do you think that validation services in your country /consistently meet clear quality standards?/ produce reliable and credible results?/

A majority of respondents agreed that this is the case, to a high extent (less than one third) or to some extent (more than one third). Free comments mentioned well-established legal frameworks, well-functioning implementation mechanisms, the links between validation and the NQFs and a robust quality assurance framework – while also noting that validation is a complex process. A minority selected ‘to a small extent’ (14%) and even less ‘not at all’.

Q6 on easy access to information and guidance on validation and Q6a on progress since 2012

58% of the responding organisations believed this was the case to a high extent (15%) or to some extent (43%), and about as many agreed that progress since 2012 has been relatively significant, with only 5% indicating there was no progress at all. The results are almost the same for the next question:

Q7 on guidance during the validation process and Q7a on progress since 2012

with 58% stating this is the case to a high extent (20%) or to some extent (38%). The share of those considering there was good progress (some or high) is somewhat lower in this case, amounting to 55% of all answers.

Q8 on how validation arrangements target disadvantaged groups (long-term unemployed, migrants, disabled persons) and Q8a on progress since 2012

Just above half of the responding organisations agreed, mostly (40%) to some extent, with a similar pattern as concerns progress. The share of those indicating ‘to a little extent’ or ‘not at all’ is the highest in relation to this effectiveness question – almost one in five respondents.

Q9 Overall, to what extent do you consider that the cost of implementing the Council Recommendation are proportionate to the benefits to individuals, the economy and society?

While a significant proportion of the responding organisations (36%) were not able to reply, a moderate majority agreed that costs were proportional, to a high extent (26%) or to some extent (27%).

Q10 To what extent do you think the Council Recommendation appropriately addresses the needs of your organisation?

Most responding organisations agreed (32% to a high extent and 35% to some extent) that the Recommendation has appropriately addressed their needs. 12% considered this was the case to a little extent with 1% going for ‘not at all’.

Q11 To what extent do you think the development of validation policies and initiatives in your country involve all interested parties?

Concerning the involvement of all interested parties in the development of validation policies and initiatives, the majority of Almost three out of four responding organisations indicated that there had been involvement to high extent (32%) or to some extent (40%). However, one in five (19%) said this had only happened to a little extent.

Q12 To what extent do you think that the Council Recommendation has contributed to /enabling individual to progress in their educational or professional development/ generating national action towards more and better validation opportunities/?

On both topics, three every four responding organisations thought that this had been the case to a high extent (24%) or to some extent (more than 40%). However one in five (19%) had a more negative view as concerns support to individual progress and one in four (26%) as concerns support to national action. Free-text comments mentioned that the Recommendation gave the impetus and some suggested a renewal and update of the Recommendation.

Q13 In general, to what extent do you think the Council Recommendation has contributed to make validation more available to people?

60% of the public consultation respondents believed that the CR contributed to make validation more available to people to a high extend (19%) or to some extent (41%), with 23% indicating that this was the case to a little extent and 6% considering there was no progress at all.

This section summarises the answers to the questions targeting respondents indicating having taken part in a validation process. As reported in Q3, 27% of the survey respondents indicated having taken part in validation. Due to the relatively low number of responses, cautious approach to the figures presented below is recommended in order to avoid inflating and misinterpreting tendencies.

Questions for validation beneficiaries

Questions 14 to 19 were only addressed to respondents who had participated in a validation experience.

Q14 How did you access the validation initiative that you used?

Almost half of respondents (44%) had found out about the validation opportunity themselves. For about one in seven validation was part of an organised initiative; as many had learned it from their employer. Only 6% took part in a validation activity after receiving information from their career guidance centre. Among the 16% that selected the ‘Other’ option, some had received a Youthpass certificate or learned it through a regional initiative.

Q15 Were you guided and supported during the validation process?

Nearly half of respondents stated they were well guided and supported, whilst 35% said that guidance and support could have been better. Only 15% had received little guidance or not at all. Free text comments reiterated the importance of guidance during validation process.

Q16 What steps did the validation process include?

A third of respondents had gone through the whole process and received a qualification or a certificate. Others passed examinations or practical tests (18%), while almost one third didn’t go beyond assisted documentation (15%) or identification (14%).

Q17 Have you received any form of financial incentive or support related to the validation process?

Only one in ten respondents (11%) had received a specific incentive to participate in validation, though as many had been somewhat supported as participants in a training programme or other project. The huge majority had no financial support, as 60% specifically stated (others preferred not to say)

Q18 Did the validation programme you took part in enable you to obtain a qualification or certificate, or part of a qualification (e.g. exemption from part of a course)?

Following validation, almost one third of the respondents (30%) had received a full qualification and another third (33%) received part of a qualification. Conversely, 20% stated neither of these was the case.

Q18a Is the qualification or certificate you obtained after undergoing validation the same that can be obtained through a formal programme (i.e. from school, college, university etc.)

This was not the case for almost half of respondents (47%), while it was the case for 38%. 6% preferred not to say it whilst 9% did not know.

Q19 Free text comment on the validation experience

A total of 23 participants provided further insights about their overall experience. In terms of ‘what went well’, respondents said the validation offered them a truly personalised learning experience with goals and a guided process. Some respondents mentioned that validation made a change of career possible. As fot he challenges, respondents referred to the complex, lengthy, and often costly procedure, as well as to difficult recognition.

Annex 3: Methods and analytical models

An external study supporting the evaluation was carried out, applying the methodology described here. The study included desk research, field research and the development of conclusions and lessons learnt.

The desk research reviewed existing literature, including studies, reports and official documentation, released by EU institutions, international organisations, national and regional authorities, stakeholders and academic research. This included in particular the 2018 update of the European Inventory on validation (Synthesis Report, associated Country Reports and Thematic Reports) and as previous updates of the Inventory (2010, 2014 and 2016), which have represented a key source of secondary information.

Other relevant sources included a Commission study on Skills Audits 211 , the one off reports by Member States on validation presented to the EQF Advisory group 212 , and the Commission study on the EU instruments supporting vocational education and training 213 . Complementary sources included European Commission publications, Cedefop resources, ECVET Secretariat resources, specialised reports from the European Youth Forum and the Lifelong Learning Platform, as well as academic papers published by the ILO, OECD and UNESCO.

Field research included key informant interviews (KII) and two expert meetings, as well as a public consultation 214 .

The contractor held 72 key informant interviews with experts from 26 Member States and five EU stakeholders’ organisations. Given the limited time available and the specialised issues addressed it was not possible to agree interviews with qualified informants from DK and LV. The table below reports on the categories of informants that could be interviewed. They include 14 members of the EQF advisory group.

Two expert meetings were held on 13 and 14 November 2019, with a total of 15 experts, invited for their personal expertise rather in representation of their country or organisation. The first meeting focused on the role of employers and other labour market actors in validation arrangements and the second one discussed how validation arrangements relate to national qualifications frameworks and the shift to learning outcomes and flexible learning pathways. The discussions have helped the contractor to assess the information obtained from other sources, namely desk research and interviews.

The public consultation was launched on 7 August 2019, in all EU official languages, and closed on 13 November 2019. It was disseminated to the relevant networks of the Commission with the request to further disseminate among their respective affiliates, beneficiaries and partners, with emphasis on stimulating responses by individuals having benefitted from validation opportunities. The questionnaire included a few questions for all respondents, some which are specifically addressed to organisations with an interest in validation and others for individuals who have used validation opportunities. Organisations were also invited to submit position papers. The 262 responses came from organisations (163 responses or 62%) and individual citizens (99 responses or 38%). The most frequent categories of organisations were public authorities (44), non-governmental organisations (38) and academic/research institutions (31).

The contractor carried out a cross-comparison of the desk research results (secondary sources), feedback from key informants, input from expert meetings, and responses to the public consultations. This supported synthesizing the results into responses to the evaluation questions for each of the five evaluation criteria and fed into the conclusions and lessons learnt.

(1)    Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning , OJ C 398 of 22.12.2012

(2)       European Pillar of Social Rights , jointly signed by the European Parliament, the Council and the Commission on 17 November 2017, at the Social Summit for Fair Jobs and Growth in Gothenburg, Sweden. Principle 1, Education, training and life-long learning: “Everyone has the right to quality and inclusive education, training and life-long learning in order to maintain and acquire skills that enable them to participate fully in society and manage successfully transitions in the labour market”. Principle 4, Active support to employment, states that everybody has “the right to timely and tailor-made assistance to improve employment or self-employment prospects”.

(3)      Commission Communication on Making a European area of lifelong learning a reality, COM(2001)678.

(4)      Council Conclusions of 12 May 2009, OJ C 119, 28.5.2009, p. 2.

(5)      Cf. Commission Communication on a New Skills Agenda for Europe : Working together to strengthen human capital, employability and competitiveness, COM(2016)381 final.

(6)      Council Recommendation of 19 December 2016 on Upskilling Pathways : New Opportunities for Adults, OJ C 484, 24.12.2016, p. 1.

(7)    Cf. Commission Communication on Building a stronger Europe : the role of youth, education and culture policies, CM(2018)268 final.

(8)      First established as a portfolio of documents in 2004 and then developed into an online platform in 2018. Cf. Decision (EU) 2018/646 of the European Parliament and of the Council of 18 April 2018 on a common framework for the provision of better services for skills and qualifications (Europass) and repealing Decision No 2241/2004/EC, OJ L 112, 2.5.2018.

(9)    First established in 2008 and then reinforced in 2017. Cf. Council Recommendation of 22 May 2017 on the European Qualifications Framework for lifelong learning , OJ C 189, 15.6.2017.

(10)      Recommendation of the European Parliament and of the Council of 18 June 2009 on the establishment of a European Credit System for Vocational Education and Training ( ECVET ), OJ C 155, 8.7.2009, p.11.

(11)    This section describes the results of the ex-ante impact assessment carried out prior to presenting the proposal for a Council Recommendation. Cf. SWD(2012)252 final : Impact assessment accompanying the proposal for a Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning. In particular section 2.2.

(12)    Cf. section 3 of SWD(2012) 252 final.

(13)      This section describes the Recommendation as adopted by the Council.

(14)    Cedefop and European Commission, European guidelines for validating of non-formal and informal learning, first edition 2009, revised in 2015 .

(15)    The Inventory was first produced in 2004 and then updated in 2005, 2008, 2010, 2014, 2018. All updates are available on the Cedefop website, here .

(16)      Cf. the 2015 Riga Conclusions , third medium term deliverable.

(17)      Cf. Commission Communication on a renewed EU agenda for higher education, COM(2017)247 final, section2.2 and the 2015 ECTS users’ guide , section 5.2.

(18)      For instance with reference to early school leaving, cf. European Commission, Assessment of the Implementation of the 2011 Council Recommendation on Policies to Reduce Early School Leaving, 2019, Figure 3.4 and pp. 76-77.

(19)      Cf. Council Recommendation of 22 April 2013 on establishing a Youth Guarantee , OJ C 120, 26.4.2013 and Council recommendation of 15 February 2016 on the integration of the long-term unemployed into the labour market, OJ C 67, 20.2.2016.

(20)      Cf. the Council Resolution on a European Union Youth Strategy 2019-2027 (2018/C 456/01), p. 4.

(21)      The main information source is the external study supporting the evaluation, resulting in the final report “ Study supporting the evaluation of the Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning”, prepared by ICF S.A and 3s Unternehmensberatung GmbH (ISBN 978-92-76-16348-0, doi:10.2767/55823). The external study has used as its main sources the 2010 ,   2014 and 2018 updates of the European Inventory of validation of non-formal and informal learning, also often directly referred to in this Staff Working Document. Another useful reference, concerning 13 EU Member States (and other 8 OECD countries) is OECD 2010, Patrick Werquin, Recognition of Non-formal and Informal Learning – Country Practices.

(22)      For the levels of development of validation in Member States, cf. Inventory 2010, Synthesis report, section 2.3.

(23)      CZ, DK, EE, ES, FI, FR, LU, MT, NL, PL, RO, SK replied positively to a specific research question by the Inventory 2010 research team.

(24)      Inventory 2010, Synthesis report, section 2.7.5 and Inventory 2014, Synthesis report, section 4.2.6.

(25)      Croatia joined the EU on 1 July 2013 and is therefore not included in this table.

(26)      It should be noted that this was also possible in five countries (BG, CY, DE, LV, PT) that did not have a validation arrangement in place, but some validation initiatives were available.

(27)      External study, Figure 10.

(28)      FR, IE, MT and UK, followed by BE (Flanders), DE, DK, EE, LT, LU, LV, NL, PT. Cf. Cedefop Working Paper No 17 , Analysis and overview of NQF developments in European countries, Annual reports 2012, p. 22.

(29)      Cedefop Working Paper No 17, p. 9. Referencing of levels could be done before the NQF was operational.

(30)      The first group included BE, CZ, LT, PT, UK, the second AT, BE, DE, ES, FI, HU, IE, LU, LV, SI, UK. External study, section 4.1.1.7, Table 13.

(31)      CZ, DK, FI, FR, IE, MT, NL, PT, SI, UK. External study, Figure 13.

(32)      External study, section 4.1.1.2.

(33)      Defined in the Glossary.

(34)      FR, HU, IE, LU, LV, NL, SE, UK. External study, Table 12.

(35)      AT, BE, IE, PL, UK. External study, Figure 8.

(36)      The direct information source is the external study supporting the evaluation, which has used as its main source of information the 2018 update of the Inventory, available here . The Inventory includes country reports that were proofread early in 2019 by EQF advisory group members representing their national authorities responsible for validation. The Inventory provides therefore a picture of the state of affairs towards the end of 2018. The external study has also carried out a number of interviews with key informants from national authorities and stakeholders’ organisations.

(37)      As explained in section 1, mentions of Member States in this Staff Working Document, which covers the period until 2018, always include the United Kingdom.

(38)      The 2018 Inventory reviews validation arrangements organised in the three areas of education and training, labour market and third sector. This disaggregation was not applied in the 2010 Inventory, so comparison is only possible on the total figure.

(39)      External study, Figure 5.

(40)      Croatia is the only Member State that has validation arrangements in the labour market and the third sector, but not in the education and training are, where they are being developed in close connection with the implementation of the national qualification framework.

(41)      Third sector: Definition in the Glossary.

(42)      External study, Figure 4.

(43)      Inventory 2018, Synthesis report, section 3.5.

(44)      External study, Annex 1, p. 139. Cf. Annex 2 to this Staff Working Document, Question 1.

(45)      Cf. Inventory 2018, Country Report Germany , p. 20.

(46)      Cf. External study, p. 28, and Inventory 2018, Country Report Poland, p. 31.

(47)      Cf. Inventory 2018, Country Report Netherlands , p. 2.

(48)      Cf. Inventory 2018, Country Report Spain , p. 7.

(49)      Cf. Inventory 2018, Country Report Lithuania , section 2.3.

(50)      Cf. External study, p. 37.

(51)      Inventory 2018, Synthesis report, p. 45.

(52)      External study, Table 6.

(53)      E.g. in France the number of people who applied for a full qualification through validation in 2016 was 5% lower than in 2015. However, it remains a high number – almost 35,000 (Inventory 2018, France Country Report, section 9.4.1), so that “the decrease in numbers might be explained by a certain level of maturity of the system”, Inventory 2018, Synthesis report, p. 34. Cf. External study, p. 39.

(54)      External study, Table 7.

(55)      External study, section 4.1.1.3.

(56)      Cf. External study, section 4.1.1.3, p. 40.

(57)      Cf. OECD 2018, Education Working Paper No 180, Viktória Kis, Hendrickje Windisch, Making skills transparent: recognising vocational skills acquired through work-based learning . In particular p. 59.

(58)      External study, Annex 1, p. 154. Cf. Annex 2 to this Staff Working Document, Question 17.

(59)      Cf. the reports on NQFs published by Cedefop since 2009.

(60)      The exception is Spain, which is expected to finalise the process in 2020.

(61)      External study, section 4.1.2.1.

(62)      CY, FI, FR, IE, IT, LT, LU, LV, NL, PL, RO, SE, UK (England Northern Ireland). Cf. Inventory 2010, Synthesis report, section 3.5.

(63)      External study, p. 61.

(64)      External study, Table 13.

(65)      External study, Annex 1, p. 147. Cf. Annex 2 to this Staff Working Document, Question 5.

(66)      Inventory 2018, Synthesis report, section 2.3.1. External study, Figure 4.13

(67)      External study, p. 64.

(68)      External study, p. 22.

(69)      Cases mentioned in Inventory 2018, Synthesis report, section 3.1. Cf. country reports for Spain and Luxembourg .

(70)      External study, section 4.1.1.2.

(71)      External study, Figure 7.

(72)      External study, Table 8.

(73)      Inventory 2018, Synthesis report, section 3.4.

(74)      External study, Annex 1, p. 147-148. Cf. Annex 2 to this Staff Working Document, Question 6.

(75)      External study, Annex 1, p. 148. Cf. Annex 2 to this Staff Working Document, Question 7.

(76)      Cf. External study, p. 104.

(77)      External study, Annex 1, p. 152. Cf. Annex 2 to this Staff Working Document, Question 14.

(78)      Cf. Cedefop research paper No 75, Coordinating guidance and validation , 2019.

(79)      External study, Table 12. Cf. also Commission, Skills Audits: Tools to identify talents , 2019.

(80)      The Validation Recommendation invited Member States to provide opportunities for a ‘skills audit’ to “individuals who are unemployed or t risk of unemployment”, Invitation 1.3.d to Member States.

(81)      External study, Table 10.

(82)      External study, Figure 8.

(83)      External study, Table 11.

(84)      External study, Annex 1, p. 148-149. Cf. Annex 2 to this Staff Working Document, Question 8.

(85)      External study, section 4.1.1.6, p. 48.

(86)      Cf. Commission Staff Working Document on the Council Recommendation on Upskilling Pathways: New Opportunities for Adults. Taking stock of implementation measures , SWD(2019) 89 final, section 2.3.3.

(87)      External study, Figure 9.

(88)      External study, Figures 14 and 15.

(89)      Inventory 2018, Synthesis report, section 3.2.

(90)      External study, Section 4.2.1.3, p.79.

(91)      From now on, the situation described refer to specific governance and support actions by the Commission and the Member States or by the Commission alone, as requested by the Recommendation.

(92)      By the end of 2019, DE, LU, LV, AT, PT, DK, PL, NO, SE had presented one-off reports on validation.

(93)      Cf. https://vplbiennale.org/.

(94)      By the end of 2019, one-off reports on validation were presented by DE, LU, LV, AT, PT, DK, PL, NO, SE.

(95)    Cedefop and European Commission, European guidelines for validating of non-formal and informal learning, first edition 2009, revised in 2015 .

(96)      External study, p. 71.

(97)    All updates are available on the Cedefop website, here .

(98)      OJ L 112, 2.5.2018.

(99)      The European Social Fund is the main financial instrument for skills related activities in the Member States.

(100)      Cf. External study, sections 4.1.3.2, 4.2.1.2, 5.2.1, and Inventory 2016 Thematic report on funding validation

(101)      External study, section 4.2.1.2, p.76.

(102)      The Erasmus+ programme supports European cooperation in education, training, youth and sport.

(103)       Structural Reform Support Programme , established to support Member States upon their request in their endeavour to design and implement reforms, including in the fields of skills, education and training.

(104)      External study, section 4.1.5.

(105)      External study, section 4.1.4.

(106)      Cf. External study, section 4.2.1.

(107)      External study, section 4.2.1.1.

(108)      External study, section 4.2.1.3.

(109)      External study, section 4.2.2.

(110)      External study, p. 80.

(111) The inventory was updated in 2010 , 2014 , 2016 , 2018 .

(112)      Inventory 2018, Synthesis report, section 3.8.

(113)      Inventory 2018, Synthesis report, section 3.3.

(114)      Cf. OECD 2010, p. 27.

(115)      Cf. External study, section 4.5.

(116)      Numerical codes between “[ ]” refer to sub-sections in section 3.2.

(117)      External study, Annex 1, p. 139. Cf. Annex 2 to this Staff Working Document, Question 1.

(118)      External study, Annex 1, p. 145. Cf. Annex 2 to this Staff Working Document, Question 4.

(119)      External study, p. 86.

(120)      External study, Annex 1, p. 142. Cf. Annex 2 to this Staff Working Document, Question 2.

(121)      External study, Annex 1, p. 153. Cf. Annex 2 to this Staff Working Document, Question 15.

(122)    External study, Annex 1, p. 154. Cf. Annex 2 to this Staff Working Document, Question 17.

(123)      With a lower score (92) for the documentation phases. External study, Annex 1, p. 145. Cf. Annex 2 to this Staff Working Document, Question 4.

(124)      External study, Annex 1, p. 153. Cf. Annex 2 to this Staff Working Document, Question 16.

(125)      The three latest updates of the Inventory suggest a positive evolution. Inventory 2018, Synthesis report, pp. 42-43.

(126)      External study, Annex 1, p. 150. Cf. Annex 2 to this Staff Working Document, Question 11.

(127)      Cedefop, National qualifications framework developments in Europe 2017 , section 3.5.

(128)      Micro-credentials can be defined as documented statements that acknowledge a person’s learning outcomes, that are related to small volumes of learning and that for the user are becoming visible in a certificate, badges, or endorsement (issued in a digital or paper format). The terminology about micro-credentials is still quite diverse, cf. OECD, Education Working Papers No 216, Shizuka Kato, Victoria Galán-Muros, Thomas Weko, The emergence of alternative credentials , in particular section 2.1.

(129)      Cf. the opinions reported at pp. 23-24 of the briefing paper “ Challenges and opportunities of micro-credentials in Europe ”, produced within the ongoing Erasmus+ project ‘ MicroHE – Support future learning excellence through micro-credentialing in higher education ’. The European MOOC Consortium is also working on a framework for micro-credentials , also an ongoing Erasmus+ project.

(130)      Cf. the report “ Digital credentialing. Implications for the recognition across borders ”, Unesco 2018, in particular pp. 27-28 and the recent articles “ HE seen as failing on social mobility, OECD expert warns ”, by Mary Beth Marklein, published on 3.2.2020 on University World News, and “ Could micro-credentials compete with traditional degrees? ”, published on 17 February 2020 on the BBC website.

(131)      Quoted from the report “ Making sense of qualifications ”, QQI (Quality and Qualifications Ireland), 2019, p. 5.

(132)      A “personalisation of a student’s learning journey”, as said in “ Micro-credential roundtable ”, note for an event organised by the New Zealand Productivity Commission in September 2019. The New Zealand Qualifications Authority has introduced a micro-credential system for tertiary education in August 2018.

(133)      External study, section 4.5.1.

(134)      External study, pp.47-48.

(135)      Cf. External study, section 4.1.3.1.

(136)      External study, p. 70.

(137)      External study, p. 70.

(138)      External study, Section 5.1.1, pp. 97-98.

(139)      External study, Section 4.2.1.1.

(140)      External study, Section 4.2.1.2.

(141)      External study, Annex 1, p. 149. Cf. Annex 2 to this Staff Working Document, Question 9.

(142)      External study, section 5.2.1, p. 106.

(143)      External study, sections 4.3 and 5.3.

(144)      External study, section 4.4.1.

(145)      External study, pp. 82-83.

(146)      External study, Annex 1, p. 150. Cf. Annex 2 to this Staff Working Document, Question 10.

(147)      External study, section 4.1.2.5.

(148)      External study, p. 86.

(149)      External study, section 4.1.1.2.

(150)      External study, section 4.1.2.1.

(151)      External study, section 4.3.1, p.84.

(152)      External study, section 4.3.2.1, p. 84.

(153)      External study, section 5.3.1.

(154)      External study, sections 4.3.4.

(155)      External study, section 4.4.3    .

(156)    External study, sections 4.5.3.1. and 4.5.3.5.

(157)      The external study mentions synergies with ECVET in Bulgaria and ECTS in Denmark, p. 63.

(158)      External study, p. 64.

(159)      Cf. European Commission, Study on EU VET instruments (ECVET and EQAVET) , 2019, section 4.1.

(160)      External study, section 4.5.3.2.

(161)      Cf. Beverly Oliver, “ Making micro-credentials work for learners, employers and providers ”, Deakin University, 2019, in particular p. 32.

(162)      External study, sections 4.5.2 and 5.4.

(163)      External study, p. 90.

(164)      External study, p. 91. Cf. above, section 5.1 Effectiveness, paragraph 1.2.

(165)      External study, section 4.6.1.

(166)      External study, section 4.6.3, p. 101.

(167)      In Italy the “principles and indications included in the Recommendation on the Validation of non-formal and informal learning of 2012 have all been taken into account in detail throughout the entire regulatory process allowing full alignment to them”, Inventory 2018, Country report Italy , Introduction.

(168)      External study, interviews mentioned on p. 98.

(169)      External study, Annex 1, p. 151. Cf. Annex 2 to this Staff Working Document, Question 12.

(170)      External study, Annex 1, p. 152. Cf. Annex 2 to this Staff Working Document, Question 13.

(171)      External study, section 4.5.2, p. 101.

(172)      In AT, HR and LU. External study, pp. 99-100.

(173)      External study, p. 100 and Annex 1, p. 151. Cf. Annex 2 to this Staff Working Document, Question 12.

(174)      External study, section 4.5.2, p. 100.

(175)      Informants from BE, BG, CZ, DE, ES, HR, IE, IT, LT, PT and SE. External study, section 4.5.3.

(176)      Informants from EL, FI, HR, SI and SK. External study, section 4.5.3, p. 101.

(177)      Informants from AT, IE, LT, MT, NL and SE. External study, section 4.5.3, p. 101.

(178)      Cf. above, sections 3.2.1 and 5.1, reply to evaluation question 1.1a.

(179)      Cf. above, section 3.2.2.

(180)      Cf. above, section 5.1, reply to evaluation question 1.2.

(181)      Cf. above, sections 3.2.6 and 5.1, reply to evaluation question 1.1.a.

(182)       Cf. above, sections 3.2.3 to 3.2.9 and 5.1, reply to evaluation question 1.1.a.

(183)      Cf. above, sections 3.2.1 and 5.1, reply to evaluation question 1.1a.

(184)      Cf. above, section 3.2.2.

(185)      Cf. above, section 3.2.7.

(186)      Cf. above, section 5.1, reply to evaluation question 1.1a, in particular Figures 5.1.3 and 4.

(187)      Cf. above, sections 3.2.8 and 5.3, reply to evaluation question 5.2.

(188)      Cf. above sections 5.1, reply to evaluation question 3, and 5.2. Cf. External study, section 5.6.

(189)      Cf. footnote 153.

(190)      Most respondents to the public consultation (80%) agreed that people should be able to validate their skills in all cases, cf. section 5.1.

(191)      On the importance of a “more systematic support to validation candidates”, cf. External study, p. 44.

(192)      Cf. mention of the evaporation effect above, section 3.2.2.

(193)      Cf. External study, p. 104.

(194)      Cf. above section 3.2.8, and external study, p.104

(195)      Cf. above sections 3.2.10 and 5.2.

(196)      Cf. External study, p. 107.

(197)      Cf. Cedefop research paper No 75, Coordinating guidance and validation , 2019. Cf. above section 3.2.7.

(198)      On the actual impact of the Validation Recommendation on national guidance policy, cf. External study, section 4.6.2, p. 100.

(199)      Cf. External study, p. 109.

(200)      Cf. above section 3.2.8.

(201)      Cf. above sections 5.1, reply to evaluation question 1.1.b.

(202)      Cf. External study, section 4.1.2.1, p. 55. In half Member States, only a part of qualifications in their NQF can be obtained through validation, cf. above section 5.1, reply to evaluation question 1.1b.

(203)      Cf. above section 5.1, reply to question1.1.b.

(204)      Cf. European Commission, Study on EU VET instruments (ECVET and EQAVET) , 2019 and the synthesis reports by the ECVET Secretariat on the Peer Learning Activities “ Units, partial qualifications and full qualifications ”, May 2016 and “ ECVET, NQFs and Upskilling Pathways ”, October 2018. ECVET is indeed mentioned as a source of inspiration in the briefing paper of the MicroHE Erasmus+ project mentioned above.

(205)      Cf. as examples of practices in Member States, Czech Republic , Finland , Hungary , Ireland , Germany , Poland , Sweden .

(206)      Cf. External study, p. 69.

(207)      Cf. External study, p. 107, p. 111 and section 5.6.

(208)    It is hosted on the Cedefop website, here .

(209)      They are available here .

(210)      A more detailed presentation f results is available in the external study, Annex 3.

(211)      Finalised end of 2018 and published in March 2019, here .

(212)      Available here .

(213)      Cf. European Commission, Study on EU VET instruments (ECVET and EQAVET) , 2019.

(214)      More details in Annex 2.