52007DC0184

[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |

Brussels, 13.4.2007

COM(2007) 184 final

COMMUNICATION FROM THE COMMISSION TO THE COUNCIL

Framework for the European survey on language competences

COMMUNICATION FROM THE COMMISSION TO THE COUNCIL

Framework for the European survey on language competences

INTRODUCTION

Multilingualism is a core value in Europe. It is part of what makes Europe unique and contributes to the richness of its culture and society. Learning languages provides people with better career possibilities, deeper understanding of their own and others’ cultures and increases their mobility.

Improving language skills in Europe is also an important objective within the drive to improve the skills and competences of the population as part of the Lisbon growth and jobs strategy. In March 2002 in Barcelona, The European Council called for further action “...to improve the mastery of basic skills, in particular by teaching at least two foreign languages from a very early age. ” They also “ called for the establishment of a linguistic competence indicator in 2003. ”[1]

Thus the Commission and the Member States undertake a range of different activities aimed at promoting good policy approaches for language learning within the Education and Training 2010 strategy.[2]

In its Communication The European Indicator of Language Competence in 2005[3] the Commission outlined a detailed strategic approach for the creation of a European Survey on Language Competence as a means to collect the data necessary to construct a European level indicator. On this basis the Council concluded in May 2006[4] on a number of key issues concerning the European Indicator of Language Competence, and stressed that a survey should be carried out as soon as possible. The Council invited the Commission to set up the European Indicator of Language Competence Advisory Board (hereafter referred to as the Board) of national experts to advise the Commission on the preparation and implementation of the survey[5]. The work of the Board has formed an important input to the writing of this communication. The Council furthermore requested the Commission to report back to the Council on the progress of work. The present Communication answers this request.

The Commission has recently presented a Communication on a coherent framework of indicators and benchmarks[6] to support the Education and Training strategy generally. The framework proposes the development of a set of 20 core indicators to cover the main domains. Creating an indicator for language competence poses particular challenges.

This Communication outlines conclusions on all the outstanding issues regarding the development of the survey highlighted in the Council Conclusion of May 2006. It invites the Council to take note of the proposed framework of the survey. On that basis the Commission, in cooperation with the Board and the Member States would be able to implement the survey. The present Communication foresees a survey to be carried out during the first six months of 2009.

FRAMEWORK OF THE EUROPEAN SURVEY ON LANGUAGE COMPETENCES

The European survey aims to provide Member States, policy makers, teachers and practitioners with reliable and comparable data on foreign language competence in the European Union. It will provide knowledge about the multilingual capacities of young people, on where good practice and performance can be found, and on progress towards the objective of improving foreign language learning.

The survey will be implemented by the Commission advised by the Board in close cooperation with the Member States.

Language Skills to be tested

In the first round of data-gathering, tests will be developed on three language skills: reading comprehension ; listening comprehension ; and writing . The Commission will take the initiative to develop instruments to cover the fourth skill speaking in time for the subsequent surveys.

The purpose of the European survey is to measure “ overall foreign language competence in each Member State ” [7]. In its final form, it should test all four language skills: reading; listening; writing; and speaking . However, some skills are easier to test than others. Tests of oral productive skills may require face-to-face tests with a human examiner and will be more complex than testing receptive skills.

The Council conclusion of May 2006 stated that for practical reasons it would be advisable, in the first round of data-gathering, for tests to be made available in the three language skills which may be assessed most readily (listening comprehension, reading comprehension and writing). The Board supports this approach.

The first round survey should therefore cover these three skills. The Commission will take initiative in time to ensure that the fourth skill, speaking , will be included in subsequent rounds of the survey.

Languages to be tested

The European survey in each Member State should cover tests in the first and second of the most taught official European languages of the European Union, namely English , French , German , Spanish and Italian .

The framework for testing will be made available for all countries that wish to ensure that tests other than in these five languages can be included in the first round of the survey as national options.

The Council Conclusions of May 2006 recommended that in the first round of the survey the test should be available “…in those official languages of the European Union that are most widely taught in the Member States, to the extent that they provide a sufficiently large sample of testees ” and that “Member States shall themselves determine which of those official languages are to be tested”[8]. The number of languages that should be assessed in the first round of the survey should therefore be limited to English, French, German, Spanish and Italian which are the most widely taught official European foreign languages in the Member States[9]. The survey in each Member State should, therefore, cover the two most widely taught of these languages.

However, the testing tool will be made available and can be used by Member States if they so wish to test competence in languages other than the two most widely taught within this list of five. The Commission will take initiative in due time, before subsequent rounds of tests, to ensure that the survey covers all the official European languages taught in the European Union.

The framework of reference

The survey should be based on an instrument measuring a continuum of increasing levels of competences from level A1 to level B2 on the scales of the Common European Framework of Reference for Languages.

The Council invited the Commission to develop the European Indicator of Language Competence and stated that “ test scores should be based on the scales of the Common European Framework of Reference for Languages ”[10].

The Common European Framework of References (CEFR) is a guideline used to describe achievements of learners of foreign languages. Its main aim is to provide framework for methods of assessing and teaching which applies to all languages in Europe. The CEFR consists of a scale with six levels for different language skills divided into three broad divisions: Basic user (A1-A2), Independent user (B1-B2) and Proficient user (C1-C2).

The six reference levels are widely accepted in the Member States as a standard for grading an individual's language proficiency. Several European countries have already used tests related to the scales of the CEFR for testing proficiency of languages.

The Board has informed the Commission that the most pertinent levels of the scale in the EU for testing language proficiencies in lower secondary education (ISCED 2), and at upper secondary education (ISCED 3) if the second foreign language is taught at that level, are the range of competences from Basic users to Independent users (levels A1 to B2). Very few pupils have the cognitive capacity to perform at a higher level than B2 at the concerned age and level of schooling. A test consisting of all the six levels would be a considerable challenge and expensive. Accordingly, it is proposed that tests should therefore be developed to cover the four levels from A1 to B2 of the CEFR.

The various levels of CEFR are wide and it takes a long time for students to progress from one level to the other. The gap between the levels (especially from levels A2 to B1) can be too large for practical use with beginners and younger students. Hence a testing instrument should be developed which enables to measure a continuum of levels of competencies between levels A1 and B2. This will also provide more precise measurements of statistical measures such as variance, median and average performances for each of the skills tested.

Contextual data to be collected

A questionnaire for students, teachers, head masters and governments will be developed, to gather contextual information that will allow analysis of possible factors which might impact on pupils’ language competences.

The intention is not only to undertake a survey of language competences but a survey that should be able to provide information about language learning, teaching methods and curricula.

Contextual information will allow relating the level of language competences of pupils in relation to possible determinants. Issues such as socio-economic background of the pupils and specific aspects like migrant status, language spoken at home; whether learning the foreign language is compulsory or voluntary; whether the teacher has formal education in teaching languages or is a native speaker; "streaming" of pupils and gender can all affect the results and data on them should be collected.

Such contextual data will address questions concerning individual pupils, schools and the type of curricula, structure of the education and training systems and governmental educational policy. Separate questionnaires addressed to sample students, teachers and school principals and to education administrators should be developed. These background questionnaires will be designed to allow for an in-depth policy analysis which may provide answers to causes and effects in developing language competences. In order to facilitate international comparison, existing concepts and classifications should be used and links to similar international surveys should be explored to allow for a multidimensional secondary analysis of the results.

Population to be tested in the survey

The "total population" of the survey, in statistical terms, should be the total number of pupils enrolled in the final year of lower secondary education (ISCED 2), or the second year of upper secondary education (ISCED 3), if a second foreign language is not taught at lower secondary education.

The "target populations" are the pupils, enrolled in schools from the total population that are taught the language being tested.

The survey will respect the general scientific standards for sampling used in international surveys to ensure that the data from the survey are reliable and can be compared between countries. The size of the sample should take into account the need for collecting contextual information in order to analyse and explain the differences in results between Member States. The survey should aim at developing reliable and comparable results with the smallest possible sample.

The Council conclusions[11] stated that the survey would gather data from a representative sample of all pupils registered in education and training at the end of lower secondary education (ISCED 2). Where a second foreign language is not taught before the end of ISCED 2, the sample of pupils should be collected from pupils at the upper secondary education (ISCED 3).

Concerning the question at which level of upper secondary education pupils should be tested, the Board advises that data should be gathered from pupils at the second year of ISCED 3. By the end of ISCED 3, some pupils have ended their education or, in the case of several countries, are enrolled in education that no longer teaches foreign languages. Furthermore, during the final year of upper secondary education, pupils and teachers are occupied with final examinations and might be less open to cooperate with the survey. The first year of upper secondary education is considered to be too early because pupils in some countries only receive teaching in the second foreign language during the second year. On the basis of the advice of the Board, the survey should gather data from pupils at the second year of ISCED 3 in the case of countries where the second foreign language is not taught before the end of ISCED 2.

Following the advice of the Board, the survey should only test pupils enrolled in schools that are being taught the language tested.

The age of pupils at the end of ISCED 2 and at the second year of ISCED 3 differs between Member States. At the end of ISCED 2 it ranges from 14 to16 years. The year of introduction of a first and second foreign language is also different in Member States. Hence the age and time during which pupils have been learning a foreign language will be different for the pupils tested in the survey. The Board stresses that this should be taken into consideration when analysing the results from the survey.

Testing instruments

Both computer based tests, using open source software, and paper and pencil tests should be made available to countries in the survey. The testing instrument should permit adaptive testing.

International experiences with computer based testing of competences have been gathered throughout recent years. DIALANG is an internet based test, currently managing diagnostic tests in 14 different European languages. International surveys such as PISA and TIMSS have made some pilot studies using computer based testing. Member States have experiences in networks on open source software. Carrying out the survey using computer based tests would be a major European achievement.

The testing instrument should be made available for countries to adapt to national needs outside the survey, and should therefore be developed according to a protocol of open source software[12].

Computer based testing represents a challenge when compared to traditional testing methods based on paper and pencil tests. Receptive skills such as listening and reading are easier to implement and deliver via computers than productive skills like writing. The Board has underlined several potential difficulties in using computer based testing linked to questions of software compatibility, the available speed of Internet connection and the computer and typing skills of pupils. At the same time there are several benefits linked to the use of computer based testing in, for example, marking and coding as well as statistical treatment of data which can be faster and less subject to error and thus more efficient. Computer based testing is also considered to be a better tool for adaptive tests where the choice of questions addressed to individual testees is tailored to the specific level of competences of the person.

The development of instruments for computer based testing is significantly more expensive than traditional paper and pencil tests, however these costs must be seen in the light of benefits of more efficient implementation of the survey and the improved basis for adapting the test to the needs of individual countries and the further development of the test for possible subsequent surveys.

On this basis, introducing computer based testing would be the optimal step forward in relation to the survey. However, the Board underlines that there are different levels of readiness in the countries concerning testing with computers. The survey should be based on alternative or complementary testing based on computers and on paper and pencil tests.

Implementation c osts of the survey

The national cost of implementing the survey will depend on the final structure of the survey. The survey should use national experiences in order to find economies of scale. The international costs will be covered by the European Union.

The level of national costs of international surveys for testing skills depends in particular on testing methods used, the sample size, the number of skills tested and the contextual questionnaires being used. The national costs of such surveys cover in particular the implementation of the test in the Member States, the setting up and running of organisational support structures, training of national and school coordinators and examiners, printing of tests, coding of test results, quality assurance procedures and translation needs.

The structure and costs of international surveys testing pupils’ skills like PISA and TIMSS, can be used as a comparable measure for costs of the survey. However, the Board considers that the survey could be more expensive than other international surveys because it will include tests of listening comprehension, and because the organisation of the survey could be more complex since there will be more subjects, more levels and more domains involved than in other international surveys.

The Board has discussed the possibility of using existing national competences, experiences and organisational structures for the implementation of the survey and thereby ensuring economies of scales (see below).

The international costs of the survey are foreseen to be covered by the European Union, within the framework of the life-long learning programme (2007 – 2013) in agreement with the Programme Committee. The international costs will cover costs related to the development of the tests, the coordination and follow-up of the Pilot tests, identification of the sample and sampling methods, the development and coordination of the full set of tests as well as analysis and reporting results.

All decisions concerning the survey that imply national costs will be taken in close cooperation with the Board and the Member States.

National organisational structures for implementing the survey

The participating countries should ensure that the necessary organisational structures are available for carrying out the survey.

Member States should take initiatives to ensure that the necessary organisational structures are available to permit the implementation of the survey and that responsibilities are defined from the very start. Most Member States have experiences from national surveys or participation in similar international surveys and could draw on such experiences. Experiences from surveys such as PISA and TIMSS can be used as a basis for planning the national organisational structures and the necessary competences and experiences can in many cases be found among national experts with experience from such surveys. Therefore, the competences and organisation of already existing services should be utilized.

Implementation of the survey

The Commission will take steps to develop the survey on language competence. Technical work should be launched in March 2007 in order to enable tests to be carried out in the beginning of 2009.

The Commission will initiate the work to develop testing instruments and to implement the survey. It will define the sampling methods and criteria for selecting participating institutions and testees, securing agreement from relevant authorities where necessary, and make sure that the quality on these issues respects the general international standard for such surveys. It will design and implement the survey, gather data and analyse and draw up the final results. The work will be carried out in close cooperation with the Board and the Member States.

The Board argues that the test should be run in the first part of the calendar year and not in the beginning of the school year. The first survey should therefore be implemented during the beginning of 2009.

In order to be able to run the test in the beginning of 2009, preparatory work should be launched in March 2007. This will lead to the development of the test and pilot tests, to be carried out in the spring of 2008. On the basis of the experiences from the pilot tests, full tests should be developed and run in the Member States in the beginning of 2009.

The Board will continue to meet throughout the whole period and work in close cooperation with the Commission on all stages of the development and implementation of the survey.[13]

CONCLUSION

The Commission invites the Council to take note of the proposed framework of the survey as presented in this Communication, concerning the following items:

- Skills to be tested

- The languages to be tested

- The levels the CEFR to be used

- The collection of contextual information

- The target and the total population to be tested

- The development of testing instruments, both computer based tests and paper and pencil tests

- The implementation of the survey

On that basis the Commission would be able to launch preparations for the survey. This will be done in close cooperation with the Board and with the Member States. Testing instruments should be developed with a view to the implementation of the survey in the beginning of 2009.

ANNEX

List of Members of the European Indicator of Language Competence Advisory Board

Country | Organisation | Title | Family Name | First name |

Council of Europe | Mr | Sheils | Joseph |

Eurydice | Ms | Baidak | Nathalie |

Austria | Österreichisches Sprachen-Kompetenz-Zentrum | Mr | Abuja | Gunther |

Belgium (Flemish Community) | Flemish Department for Education and Training - Curriculum Division | Ms | Raes | Nicole |

Belgium (German Community) | not yet nominated |

Belgium (French Community) | Service général de pilotage du système éducatif | Mr | Woolf | Alain |

Bulgaria | not yet nominated |

Cyprus | Ministry of Education and Culture | Ms | Tofaridou | Efrosyni |

Czech Republic | Institute for Information on Education | Ms | Kramplova | Iveta |

Denmark | Ministry of Education, International unit | Ms | Andersen | Pernille Skou Brønner |

Estonia | National Examination and Qualification Center | Ms | Sõstar | Kersti |

Finland | Finnish National Board of Education | Ms | Mustaparta | Anna-Kaisa |

France | Ministère de l'Education Nationale, de l'Enseignement Supérieur et de la Recherche | Mr | Monnanteuil | François |

Germany | Deutsches Institut für Internationale Pädagogische Forschung (DIPF) | Mr | Klieme | Eckhard |

Germany (substitute) | Deutsches Institut für Internationale Pädagogische Forschung (DIPF) | Mr | Hesse | Hermann-Günter |

Greece | Ministry of National Education and Religious Affairs | Mr | Papadakis | Nikolaos |

Greece | Greek Unit of the Eurydice Network, Ministry of National Education and Religious Affairs | Mr | Papamanolis | Nikos |

Hungary | ELTE University - National Institution of Public Education | Ms | Major | Eva |

Hungary | Ministry of Education | Ms | Kádár-Fülöp | Judit |

Ireland | Centre for Language and Communication Studies | Mr | Little | David |

Italy | Directorate-General for Foreign Affairs, Ministry of Education | Ms | Di Nicuolo | Giulia |

Latvia | Ministry of Education and Science | Mr | Mankovs | Leonīds |

Latvia | The Centre for Curriculum Development and Examinations | Ms | Muceniece | Gundega |

Lithuania | Education Development Centre, Ministry of Education and Science | Ms | Jariene | Raimonda |

Luxembourg | Ministère de l’Education et de la Formation professionnelle | Mr | Fandel | Jean-Claude |

Malta | Ministry for Education, Youth and Employment | Mr | Camilleri | George |

Netherlands | Hesta Advies - en Vertaalbureau | Mr | Molenaar | Peter |

Poland | Ministry of National Education | Mr | Poszytek | Pawel |

Portugal | Faculdade de Psicologia e Ciências da Educação, Universidade de Lisboa | Ms | Peralta | Helena |

Romania | National Institute for Educational Sciences | Mr | Nasta | Dan Ion |

Slovakia | National Institute for Education, Department of foreign Languages | Ms | De Jaegher | Darina |

Slovenia | waiting for new nomination |

Spain | Ministerio de Educación y Ciencia - Instituto de Evaluación (IE) | Ms | Tovar Sánchez | Carmen |

Sweden | Skolverket - Swedish National Agency for Education | Mr | Lagergren | Tommy |

United Kingdom | Strategic Analysis and Data Services Group - Department for Education and Skills | Mr | Leman | Steve |

United Kingdom (Wales) | University of Swansea | Mr | Meara | Paul |

United Kingdom (Scotland) | Scottish Qualifications Authority | Mr | Van Krieken | Robert |

[1] Presidency conclusions, Barcelona, part I, Paragraph 43.1. March 2002.

[2] A new framework Strategy for Multilingualism, COM (2005) 596 final

[3] The European Indicator of Language Competence COM (2005) 356 final

[4] Council conclusions of 18 and 19th May 2006 on the European Indicator of Language Competence (2006/C 172/01)

[5] The Board was set up by the Commission in a Commission Decision of 26th October 2006. The Board has had two meetings during 2006.

[6] A coherent framework of indicators and benchmarks for monitoring progress towards the Lisbon objectives in education and training COM (2007) 61 final

[7] COM (2005) 356 final

[8] Council Conclusions (2006/C 172/01)

[9] Eurostat UOE.

[10] Council Conclusions (2006/C 172/01)

[11] Council Conclusion (2006/C 172/01)

[12] The Centre for research on lifelong learning (CRELL/the Joint Research Centre, Ispra) is currently carrying out a research and development project on open source computer-based assessment tools for skills assessment taking into account on-going research initiatives in this field.

[13] The financial and human recourses of the foreign language indicator survey have already been outlined in the financial annex of COM (2005) 356 final