Developments 2011

At the closing conference of the 2008-2011 ECML projects, the RelEx project group presented its publication Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching assessment (CEFR) – Highlights from the Manual. This publication was well received by the participants, a number of whom singling out the project as extremely relevant in a time when people’s foreign language proficiency levels have become very important for their chances in life. Such levels must be assessed through valid testing, as the publication contends.

Project members Gilles Breton and José Noijons gave a workshop on the above-mentioned linking issues. Participants expressed their satisfaction with the methods put forward in the publication. Some of them argued that the ECML might like to study possibilities for such linking procedures to be introduced in participating member countries through workshops in these countries.

Expert meeting, 10-11 February 2011

In February 2011 the ClassRelEx team met at the ECML in Graz to evaluate the November 2010 workshop on linking classroom tests to the CEFR. From the feedback of participants it became clear that participants were (highly) satisfied with the workshop and are eager to disseminate what they have leant in their own environment.
The project team also finalized the Highlights from the Manual on linking examinations to the CEFR. These Highlights will be published by the ECML within the coming six months and will be available in print and on the web. A glossary with technical linking terms will also be provided.
The project team evaluated their work in the past year, with special reference to the ECML context.
Lastly, the team members discussed the new ECML Medium Term Programme for 2012-2015. The team expressed some concern about the relatively sparse attention that is given to assessment and evaluation in the programme. The project team members have expressed an interest in continuing their cooperation in the new Medium Term Programme.

Project developments 2010 and workshop in Graz, 24-26 November 2010

At the ECML in Graz a workshop was held on Training in Classroom Assessment related to the Common European Framework of Reference for Languages (CEFR). The three-day workshop in November 2010 was attended by over 30 participants representing a corresponding number of member-states. The workshop was given by the RelEx project team with the professional support of the ECML staff. The aim of the workshop was to provide teacher trainers with training and training materials in relating language tests and examinations to the CEFR based on the stages and procedures specified in the final version of the Council of Europe Manual. Teacher trainers have been trained in recognizing adequate linking procedures and in using some of the tools to relate (classroom) tests to the CEFR. To this end highlights from the Manual were provided to participants. These highlights contain the most important parts of the Manual, with technical psychometric information reduced to a minimum. The workshop has emphasized that linking tests and exams to the CEFR is a specialist job, but that teachers should be able to recognize tests that have been properly linked and be able to judge whether claims as to CEFR levels are valid.

Information for workshop participants - please click


Expert meeting

The project members of the ClassRelEx project (Classroom Assessment related to the Common European Framework of Reference (CEFR)) met at the ECML in Graz 25-26 March 2010 to discuss and review work done in connection with the Workshop to be held in November 2010.

The project member also reviewed a first draft of a Guide to the Manual for linking tests and examinations to the CEFR. This Guide is aimed at a more general public of language specialists; it highlights the important issues discussed in the Manual and gives guidance to teachers on CEFR-related classroom assessment.


Expert meeting, 29-30 October 2009

The RelEx project members have had a meeting to evaluate the past workshop on linking language exams to the CEFR. As was found in the feedback that participants provided, the workshop has met its aims and some promising follow-up activities have been carried out. One such activity is the creation of the so-called Prague Network in which a number of testing organizations in Central Europe intend to cooperate in developing valid final FL examinations properly linked to the CEFR. There will be more information on this initiative on the RelEx website soon.

Participants in the workshop asked for ready-to-use materials on the linking process. The project members are now in the process of preparing an introductory document for those interested in the linking procedures as laid out in the Manual. The document is aimed at policy makers, testing organizations, testing professionals and teacher trainers.

The RelEx project members have also prepared for a new workshop aimed at teacher trainers. The aim of this workshop is to assist teacher trainers in developing training materials enabling teachers to recognize, choose and prepare for classroom assessment that is related to CEFR.


RelEx Workshop, 13-15 mai, 2009

From 13 to 15 May at the European Centre for Modern Languages in Graz, Austria, a workshop was held on linking examinations to the Common European Framework of Reference for Languages (CEFR). Over 30 participants took part in the workshop, representing some 30 European countries/regions. The workshop aimed at making participants realize the importance of validated claims of links of examinations to the CEFR. The workshop illustrated how such claims can be validated and partcipants engaged in examples of procedures to arrive at valdiated claims.
In lectures by the RelEx project team extensive reference was made to the final version of a Manual on Relating Language Examinations to the CEFR (published by the Council of Europe). During the workshop participants were introduced to the various stages in the linking process: familiarisation, specification, standardisation, standard-setting and validation. In groupwork participants explored and discussed topics and challenges in the linking process.

Participants have indicated that they have found the workshop stimulating and that they hope to carry the linking process forward in their respective countries and institutes. It has been realized that linking examinations is a complex process for which much expertise is needed. The workshop has encouraged participants to share experiences and to seek expert advice from the Council of Europe in the area of linking examinations to the CEFR. 

Valencia workshops

Two workshops on Relating language examinations to the Common European Framework of Reference for Languages (CEFR) were organised jointly by the APEOICVA (Association of Official School of Languages Teachers in the Community of Valencia, Spain) and the ECML (European Centre for Modern Languages) of the Council of Europe in Graz. The workshops were held in Valencia on 6-7 February 2009 and on 27-28 March 2009. The workshops were led by Waldek Martyniuk, director of the ECML in Graz, Austria and José Noijons, Senior Consultant at Cito, the Netherlands Institute for Educational Measurement. Both experts have an extensive experience in relating exams to the CEFR, in issues related to the CEFR in general and in exam construction in particular. 


Some 50 – 60 teachers and other APEOICVA officials attended each of the workshops. Most of the participants were actively engaged in teaching language classes, most of them were active as examiners as well and some of them were item writers. A few of the participants were responsible for the development of test administration procedures and marking schemes, grading systems. Some of the workshops have been videotaped in order to help disseminate the experiences in the workshops.

As is borne out by the programmes of the events and the accompanying materials, the workshops focussed on the five phases in the linking process as exemplified in the Manual for linking examinations to the CEFR: familiarization, specification, benchmarking, standard setting and validation.

For the familiarization process the participants familiarized themselves with the content of the CEFR, its background and its role. Also, the important role that the Council of Europe has played in developing the CEFR was dealt with. The participants were then given tasks to carry out to help them develop a good understanding of what the CEFR stands for. For the specification phase samples of the APEOICVA exams in English were specified in terms of the CEFR (levels). Participants came to realize that linking exams to the CEFR is only possible when tests can be associated with specific descriptors in the CEFR. For the standardization phase (benchmarking) exemplars of possible methods to link (speaking and writing) performances to specific levels in the CEFR were touched upon and practiced with. Similarly various methods of setting standards in tests of receptive skills (listening and reading) have been discussed and some sample procedures were carried out. It was emphasized that standardisation is a complex process that may need statistical support. Also the fact was noted that standardization is a group process in which experts together reach agreement on standards. Finally for the validation phase participants had a close look at the quality of the instruments, their validity as far as content was concerned, the robustness of administration procedures, equivalence of exams across languages and across years, and equity issues. It was concluded that if tests and the procedures to administer them do not meet given international standards, linking to the CEFR would seem to lack substance.

A number of interesting points were raised relating to the expertise of those engaged in the examination process. Even though teachers may have been trained in producing tests, marking them and interpreting results, certainly in the case of high stakes tests test development may need to be in the hands of professionals that have been specifically trained on the  job of developing exams. A first step towards this is to have trained coordinators whose job it would be to on the one hand disseminate testing expertise and on the other hand coordinate the testing process for a particular (group of) language(s) at a particular level. For these coordinators to have sufficient expertise to do their job continued (CEFR-related) training in test development may be helpful.

As was pointed out above, the use of statistics is absolutely necessary to guarantee that the exam process meets international standards and relevant links with the CEFR can be made. Institutes will need to employ statisticians to help analyse field tests and the administration of real exams in order to make sure that reliable and valid tests are produced and exams have comparable difficulties from year to year and from language to language.  

The workshops have been conducted in a very pleasant and warm atmosphere in which it was possible to develop, share and challenge opinions.

Link to presentations given at the seminar

Expert meeting RelEx project team 5-6 March 2009

On 5-6 March 2009 a meeting of the RelEx project team took place. The project members are José Noijons (coordinator), Jana Berešová, Gilles Breton and Gabor Szabo. The director of the ECML took part in a number of the team’s discussions.

The project team agreed in their workshop scheduled to take place 13-15 May 2009 to closely follow the recently published final version of the Manual on Relating Language Examinations to the CEFR. In the Manual five procedures in the linking process are distinguished: (1) Familiarisation, (2) Specification, (3) Standardisation training/benchmarking, (4) Standard setting and (5) Validation. These five procedures will all be introduced, illustrated and discussed during the three-day workshop.

The workshop will be aimed at experts working in national examination bodies in their countries of origin. This does not necessarily mean that participants need to have extensive statistical expertise. The workshop is aimed at making participants realize the importance of validated claims of links of examination to the CEFR. The workshop will illustrate how such claims can be validated and will engage participants in examples of procedures.

Participants will be asked to do a limited amount of preparatory work on one of the exams produced in their own country and to bring samples of examinations to the workshop.


As a piloting event to the RelEx project a seminar organised jointly by the APEOICVA (Association of Official School of Languages Teachers in the Community of Valencia, Spain) and the ECML (European Centre for Modern Languages) of the Council of Europe in Graz is taking place in Valencia, Spain, on 6-7 February 2009.

Link to programme of the seminar

Link to presentations given at the seminar





Bookmark and Share