Assessment 2015

No More Levels - Greenwich Feb 2016

Presentation (PPT)
Handouts (zipped folder)

Life after levels keynote - Nov 2015

Assessing progress in language learning at Key Stages 2 and 3

Framework documentation

1 The 10 Steps framework for KS2 and KS3 (v3 Jan 2015)
2 The KS2 descriptors (with ‘what to look for’ evidence descriptors applied to our KS2 Spanish Scheme of Work).
3 Materials for introducing the framework to a languages department:
i) Overview document
ii) Card sort (listening strands 1 and 2)
iii) Question prompts (and suggested answers)

Context

The context in which this assessment framework has been produced is fundamentally a local school context. In my organisation, as in all secondary schools in England, we are in a situation of some considerable uncertainty. KS2 and KS3 National Curriculum levels have been officially declared obsolete by the current government for all but the KS2 external examinations for summer 2015. Schools are free to determine their own approach to assessment, data collection, achievement monitoring and reporting to parents, but they are under pressure to do so as they must publish their assessment policy for KS3 on their own school websites by September 2015. The situation is further problematised by the changes at KS4, where GCSE letter grades will be replaced by number grades for first examination in some subjects in 2017 and in languages from 2018. In languages in particular, we have the further issue that a progression framework for learning in our subject must take account of learning at KS2, and it cannot therefore be produced in isolation assuming an ab initio starting point in Year 7.

I started work some time ago, trying to produce a progression framework for language learning, assuming a starting point in Y3 and a notional end point in Y9, whilst envisioning at the same time the two subsequent years that will culminate in GCSE. Initially I waited for a whole-school steer, but ultimately it became clear that we were being asked as individual subject specialists to describe progression within our own subjects, before any decisions are taken about how to integrate all of this into one coherent system of measuring and reporting progress at a whole school level.

What follows, therefore, is an attempt to do exactly that. I’ve chosen the number of progression points to describe, and the nomenclature (both of which may therefore have to change!). I have shared earlier versions of the framework on an ad hoc basis with several colleagues and subsequently revised the descriptors in light of their comments. From early January in the schools in which I work we will be looking carefully at the framework, trying to provide the supplementary sections currently lacking (i.e. the list of themes, the lists of non-cognate vocabulary for each language) as well as re-calibrating our current milestone assessments to align them with it. There will no doubt be many more revisions along the way.

I therefore share this framework in the spirit of collaboration with other linguists in the same position as we are, tasked with creating a new assessment framework based on the requirements of the new Programmes of Study at KS2 and KS3. I do not offer it up as the finished article, and not even as a finished article, but as one example of a work in progress, which I hope can be of some use to others.

Rationale

The framework has the look and feel of a proficiency assessment framework, providing illustrative descriptors of what learners can do when they apply their knowledge in communicative tasks. As such the descriptors describe emerging linguistic competence as evidenced through performance in each of the skills, as well as their applied use of grammatical structures. The supporting vocabulary strand describes a repertoire of knowledge that learners draw on, but it is also framed within the context of the act of either understanding or producing that vocabulary. In most respects, it avoids too close an orientation to the specific content of a course. It does not specify anything that could be interpreted as the week’s or term’s work.

However, in the detail concerning the number of words, or the listing of specific structures, or the number of topics or length of interaction time, the aim has been to give teachers more specific guidance about the broader features of progression within classroom language learning, in which breadth of topic coverage, length of utterance or interaction or extent to which written language is generated independently (to name just a few) are all very significant indicators of progress.

Achievement assessment (measuring the degree to which learners have learnt / remembered what we have taught them this week or this term) is important to teachers, and can be motivating to learners too, as it feels concrete and related to their most recent classroom experience. Ideally there need not be a huge gap between these two types of assessment. As the CEFR notes:

“To the extent that an achievement assessment tests practical language use in relevant situations and aims to offer a balanced picture of emerging competence, it has a proficiency angle. To the extent that a proficiency assessment consists of language and communicative tasks based on a transparent relevant syllabus, giving the learner the opportunity to show what they have achieved, that test has an achievement element.” (CEFR, p.183-4)

One thing I have tried to avoid is providing descriptors that can be reduced to a list of ‘things to do to reach the next level’. There is a fine line between description and prescription. That is not to say that teachers and learners should not look at the descriptors for the next step to help inform their target-setting, but that the descriptors do not provide an itemised checklist of things learners need to ‘do’ to move up to the next level. Each step describes the features of competence representative of approximately 30 hours of learning, which is approximately half a year of language lessons, such that drafting a weekly, half-termly or termly target directly from the next step’s descriptor would not be fit for purpose.

Existing frameworks

1 The Common European Framework

In the development of the illustrative descriptors in this framework, I drew heavily on several key documents that already describe progression in language learning. The first of these is the Common European Framework, which I used in two key ways. First, I borrowed the suggested guided learning hours (GLH) for the first three levels A1, A2 and B1 to inform the overall shape of progression. There are some variations in the suggested GLH, but I used the Alliance Française model as it refers to one of the main foreign languages taught in schools in England.

	GLH
A1	60-100
A2	160-200
B1	360-400
B2	560-650
C1	810-950
C2	1060-1200

For reference, there are other GLH for comparison here.

The GLH were helpful in deciding the approximate progress we can expect during each key stage. All primary and secondary schools vary in their allocation of curriculum time for language learning, but here I have taken as a starting point 30 minutes per week in Years 3 and 4, 60 minutes in Years 5 & 6, and 150 minutes per week across KS3 and KS4. (This equates to 3 x 50 minute lessons across KS3 and KS4. This is a rough average. In fact, our dual linguists have less and our single linguist classes have slightly more than this.)

Year	Hours of tuition	Cumulative hours	CEFR Level
3	19	19
4	19	38
5	38	76
6	38	114	A1
7	95	209	A2
8	95	304	A2+
9	95	399
10	95	494	B1
11	95	589
12	190	779	B2
13	190	969	C1

We can see from the table that, with this number of GLH, we might reasonably expect learners to achieve at least A1 competence by the end of KS2, and at least A2 by the end of Y8 (assuming the curriculum allocation hours given), making significant progress towards B1 but not reaching it by the end of Y9. I used this information to peg the key progression points A1 and A2 within the new framework.

For further comparison, it is worth remembering that the CEFR levels have been equated to other frameworks, including former national curriculum, the Languages Ladder levels, and current GCSE levels. Those equivalences are as follows:

CEFR Level	Languages Ladder	NC	current GCSE / AS / A2	new GCSE
A1	1-3	1-3	Entry 1-3
A2	4-6	4-6	F-D	1-3
B1	7-9	7-8 + EP	C-A*	4-8
B2	10-12		AS/A	9
C1	13-15		Degree
C2	16-17		Degree +

NB: The positioning of the new GCSE grades has been done as a result of information received about the alignment of the new grades with current (i.e. that the current C = the new 4). I have aligned the new 9 with B2, as result of comparing the new GCSE assessment objectives with the CEFRdescriptors.

The second use of the CEFR was a borrowing and adaptation of many elements of its numerous illustrative descriptive scales, to be found in Chapters 3-5, from the overall self-assessment grid (p.26-27) to the more detailed descriptors (p.58 – 129). In addition I found it helpful to explore the many accredited and registered versions of the European Languages Portfolio, particularly the SCILT primary version.

2 The Languages Ladder and the former National Curriculum levels

Both the Languages Ladder and the former NC level descriptors have informed the descriptors in this new framework, which is unsurprising as they themselves borrow heavily on the CEFR. However, as the starting point for the new framework had to be the statutory Programmes of Study at KS2 and KS3, additional strands have been included. In addition, there has been an attempt to address some of the difficulties in the former NC level descriptors.

The National Curriculum levels were initially supposed to give you an idea of how your child was doing at the end of each key stage. These were supposed to be approximations reached by teacher judgment considering all their work over the previous years, and based on reasonable expectations of achievement based on the national profile. They were never intended either to be used to mark individual pieces of work, assigned to single assessments or to set lesson objectives! Over time the drive for measurement has meant that NC levels have been put to increasingly reductionist use, including in most schools the use of invented ‘sub-levels’, over which there is the haziest understanding and consistency!

Furthermore, there were several difficulties in the formulation of the descriptors themselves. First, they do not describe anything close to equal progression steps in learning and yet, teachers and departments are expected to produce linear progress of, on average two sub-levels of progress per year, two full levels of progress across Key Stage 3. This is problematic in languages where the distance between level 4 and 5, say, is far greater than between level 3 and 4. Second, the need to apply levels to individual tasks and tests inevitably meant a reductionist interpretation of each level, such that a pupil producing a piece of work with an example of two tenses is awarded a level 5 or a text book passage with a smattering of three time frames is instantly branded level 6, even where other features (e.g. the speed of delivery, the predictable nature of the material, the straightforward sentence structure) might make it more indicative of a lower level overall. These distortions have not been helpful. The preoccupation with tenses as the main indicator of linguistic progression has been problematic and there has been an effort to address this within the new framework. Finally, it has been extremely unhelpful to describe achievement at the upper end of the spectrum only attainable in reality by those with native level ability. Levels 8 and EP have been virtually unused in most schools at KS3 for assessment and reporting of progress within modern languages.

3 The draft APP (Assessing Pupils’ Progress) Assessment Guidelines

Although never published, there was an attempt in 2009 to create a languages version of the APP guidelines. These are still in use in many schools in English and Maths and have formed the basis of a new assessment framework in several schools that I am aware of. I referred also to the wording in these descriptors in the formulation of those in this framework. Believing that the document is no longer confidential, I include a link to it here.

4 The renewed KS3 Framework

This non-statutory revised version (2009) of the original KS3 Framework for Languages (2003) is under-used in my view. No money was provided for its dissemination and it therefore got rather lost. As with many previous useful documents, it has been buried in the archives, so I’ve uploaded it here.

5 The Professional Development Consortium in Modern Foreign Languages (PDC in MFL) assessment framework

The PDC’s work has been a highly positive development for teachers of languages. It has set out to distil from empirical research some core principles of language teaching and learning. Following on from this, the group has development an alternative assessment framework, which I have drawn on heavily, particularly on some of the features of its supporting strands. I would encourage any department to take time to read the material, case studies and view the classroom video on the PDC site.

6 The KS2 Framework

It would have been nonsense to generate descriptors for progression within language learning at KS3 without using the KS2 Framework to underpin the statements. Whilst non-statutory, it remains the best source of guidance to teachers planning their curriculum, their teaching and their assessment at KS2.

The 10 language learning steps

i) Framework structure

The 10 steps framework sets out to define progression in terms of the attainment targets in the new Programmes of Study at KS2 and KS3. Each ‘step’ consists of illustrative proficiency descriptors, a set of ‘can do’ statements. The PoS attainment targets have been set into strands, where they refer to sufficiently different aspects of linguistic proficiency. There are two strands for listening, two for speaking (with two further sub-strands for each) that correspond to interaction and production respectively, three for reading, two for writing, one for grammar and one for vocabulary. The intention is not that each strand would be reported on separately, but that at all times a more holistic ‘best fit’ judgement would be made for each skill, and an overall level.

Recognising (with some reluctance) that in the current climate the drive for the linear measurement of progress is a dominant factor, and that achievement-focused assessment is here to stay, the creation of this new framework has attempted to describe roughly equal steps, whilst taking into account the different time allocations at KS2 and KS3. The result is that Steps 1-4 are a little further apart (representing approx. 114 hours of learning at KS2), Steps 5 – 7 are a little closer together (representing the next 142.5 hours) and Steps 8 – 10 represent a further 142.5 hours. The KS3 steps are therefore fairly equally spaced with 47.5 hours of teaching (or six months of lessons) between them.

ii) Expected progress and predictions

In each academic year we must anticipate a spread of achievement. The majority of learners whose entry point is Step 4 might expect to achieve Step 7 at the end of Y8, with the most able achieving Step 8. Those same learners would progress to Step 9 by the end of Y9, with the most able achieving Step 10. The range of expected rates and levels of progress are more easily captured in tabular form:

Year	3	4	5	6	7	8	9
Steps range KS2 & KS3	1-2	1-3	2-4	2-4(+)	3-6	5-8	6-10

By the end of KS3, it should be possible to generate an expected range of future performance in the new GCSE exams (first teaching September 2016 to current Y8). This will, of course, be at best highly speculative, until we have had the first cohort through the examination in 2018, but we will nevertheless still need to generate these sorts of predictions:

Year 9 Step	6	7	8	9	10
new GCSE predicted grade range	1-4	4-6	6-8	7-9	8-9

iii) Additional requirements

To enable the framework to function as a useful assessment tool within a school, several additions are required. These would most likely form key parts of the scheme of work:

1) a list of 200 non-cognate words for each language taught
2) an overview list of the main topics and sub-topics to be learnt throughout the key stage
3) a list of the types of text to be included in listening and reading tasks
4) a list of the grammatical structures to be learnt, including when they are first introduced, subsequently re-used, and when independent use is expected. Note that the grammar strand included within the framework itself is not as generic as it might at first sight appear, being informed by the KS2 and KS3 Spanish schemes of work in use in our schools.

iv) Reporting documents

It is too early as yet to know how my school will want us to report to parents and students. However, one of our partner primary schools is currently reviewing its reporting documentation. After a meeting with the languages co-ordinator there, I devised an adaptation of her school’s current learning journal documentation, to fit with the new KS2 PoS and this framework’s descriptors. This is still very much in draft, but I’ve uploaded it here as an example.

As I’ve mentioned, this is very much a working document and it will no doubt be subject to much change and revision! In the interests of collaborative working, please do feel free to contact me with any ideas and thoughts, or share alternative models.