Striving for a personal best – who judges?

In this post I’m going to look at some of the issues surrounding a form of assessment you’re probably familiar with, but not with the terminology that goes with it: ipsative assessment. So what is ipsative assessment? It’s a means of assessment that measures the student’s progress, not by comparing to some external (and often arbitrary) standard, but by comparing the student’s achievement now to their achievement in the past. It asks the questions: Did I improve? By how much? It avoids the questions: What mark did I get? Did Fred do better than me?

This shift from external validation to internal comparison is an important one, and has a number of implications. Firstly, when done well it moves the focus towards personal growth and development. This, to me, is what education should be about. Arguably it’s the only measure that really matters long term, although the secondary effects of gaining qualifications and meaningful employment are also important. Education in this sense vastly exceeds the education-as-job-training model, developing citizenship as well as knowledge. My model of education always reminds me of a line from the film Dances with Wolves, when Kicking Bear (Grahame Greene) is talking to John Dunbar (Kevin Costner): “I was just thinking that of all the trails in this life there is one that matters most. It is the trail of a true human being. I think you are on this trail and it is good to see.” This idea is not new and goes back to Ancient Greece, where the responsibilities of citizenship were deeply connected to everyday life.

coaching word cloud

Coaching word cloud

Ipsative assessment in widespread in sports, and is often practised in conjunction with a coach – it’s the process behind the concept of a ‘personal best’, where achieving a personal best is a cause for celebration regardless of the result compared to others. I have my own boat (a GP14) and sail at a local club. It’s not a competitive boat. That’s not an excuse – the boat was built in 1973 and still has the same sails (which were old then) as when I bought the boat around 2000, so I’m not going to be winning any club races, even if I were a better sailor 🙂 . But it doesn’t matter: what matters is did I have a good race? How well did I spot the wind shifts? How well did I round the marks? In short, how am I sailing in this race compared to my own skills? Which, of course, is the only thing I can actually control. In this way, ipsative assessment is ‘performance in practice’, and as such is a prime example of authentic assessment.

The school system here in the UK uses a measure called ‘value-added’, which looks at the change in scores between various stages of schooling, with examination or test scores being the primary accountability measure. The downside is that if this measure is used to judge schools rather than record their performance then there will be pressure to game the system, which means that value-added isn’t measuring what it’s supposed to. In addition, I recall reading a blog recently where a teacher was criticising value-added because it assumed that the class that were tested in year 6 contained the same children that were tested in year 1. Their particular school had a high turnover of children because of local circumstances so the assumption didn’t hold. How on earth can you measure progress without tying the data to an individual? Surely without that link value-added has no value at all?

What I like about ipsative assessment is that the attention is focused on what has been achieved rather than what has not. It also gives us an additional chance to engage learners with taking responsibility for their own learning and that’s crucial for ipsative assessment to be effective, although achieving it can be problematic. When my daughter was taking her degree each assignment had a self-assessment sheet that was a copy of the one used by the tutors. The students were supposed to judge their own performance and then compare it to the tutor’s sheet when the work was returned. My daughter, despite many conversations about why self-evaluation was useful and what the educational purpose was, would simply tick the middle box all the way down. In effect, she deferred to an external authority to judge her work.

Conceptually, there is also a link to assessment for learning (A4L). While A4L allows the teacher to gauge student knowledge it can also be seen as ipsative assessment for the teacher that then feeds into their reflective practice.

A key question is how can we formalise ipsative assessment without losing the ethos behind it? We need structures and procedures to mediate and support the process, but the last thing education needs (especially schools-based education in the UK) is another accountability stick with which to beat the teachers. Firstly, if the process is simply a tick-box exercise then it’s not being allocated the importance it needs, and neither student nor teacher will take it seriously. Secondly, it’s vital that the process is student-owned. The student must be taking part in an active way with evaluating and processing their ipsative feedback for them to get the gains it offers. As Hughes has pointed out, the student needs to be the main participant, not the teacher.

In a previous post I described Jeremy Levesley’s approach to assessment, and this could fit quite nicely with ipsative assessment. Suppose we use Prof. Levesley’s approach so that we’re getting to a mark or grade quickly and then use the freed time to put more effort into the ipsative process? We get a mark to meet our accreditation needs (and those students who will still fixate on the mark above all else), and we get to develop learner independence and the self-assessment capabilities of our students. It seems like a win-win, but would a hybrid approach work, or are we just contaminating the ipsative process? I believe it could be done if the academic systems we choose to adopt within our course reinforces the practices we wish to see in our students.

The reason I think ipsative assessment isn’t more prominent in education at the moment is the relentless focus on education (and students) as a product, rather than education as a process, as Cory Doctorow recently observed in his wide-ranging video on openness, privacy and trust in education. And that’s the wrong focus. Why train students to perform in systems that are so unrepresentative of the world beyond the campus when we could teach them to judge themselves and extend beyond their personal best?

The Culture of Student Feedback

Feedback is a big issue in education – how much students get, and how (or if) they use it. Feedback features in the annual National Student Survey, and no matter how good the institutional results it remains one area that is frequently marked down.
David Boud and Elizabeth Molloy published a recent paper  on rethinking models of feedback that I found quite interesting. First, I’ll explain a little about the paper itself and then my thoughts on it, particularly as it relates to institutional culture.
Boud and Molloy distinguish between two models of feedback: the traditional model of feedback, which they call feedback mark one, and their model, which they call feedback mark two. They talk about traditional models by comparing them to the process in biological and engineering systems, i.e. that ‘information’ acts on a system in order to change the output. In educational terms, this is feedback as ‘telling’, feedback as a behaviourist pedagogy, and assumes that not only is the information given sufficient to actually produce a change, but that it’s unambiguous and that students will actually make use of it.
Feedback mark two is more of a developmental process and less of an add-on extra to an assessment. The aim is to transition students to become ‘agents of their own change’ so that they seek out information for improvement themselves rather than merely respond to it. Students become aware of what quality performance and feedback look like through dialogue, and through this process students develop the capacity to monitor and evaluate their own learning. Assessment tasks are designed so that students are engaged over a period of time so the generation, processing and use of feedback happens over a number of cycles. In other words, it’s making the receipt of feedback and reflection on it, (and the development of skills to do that) explicit.
Essentially, three elements work together: learners (and what they bring to learning), the curriculum (and what that promotes), and what Boud and Molloy call the ‘learning milieu’, which is the interplay of staff, students and the learning environment. Feedback shifts from being an act of teachers to an act of students (with teacher support), from a process involving a single source to multiple sources (with a corresponding shift from an individual to a collaborative act), and from an isolated event to a designed sequence of events.

My thoughts

Feedback mark one isn’t sustainable. First, it absorbs a huge amount of time and resources, but its effectiveness is questionable in actually influencing student behaviour and improving student outcomes, since students may simply look at the mark and ignore the carefully considered comments. Secondly, if there is a dual emphasis on student improvement and on improving NSS scores then the problem is that students don’t recognise the feedback they receive as actually being feedback. Thirdly, mark one feedback isn’t fit for purpose since it doesn’t equip students for life post-graduation.
Feedback mark two is, to my mind, a desirable development. It’s better pedagogically because students actively use the feedback. It’s better for the students because they are developing (and using) skills that will help them in lifelong learning and in employment after graduation. It’s better for the institution because it reduces the workload of academics (allowing more time for research, professional development, or improving their teaching in other ways), and because it improves the quality and performance of their graduates. The NSS is actually a barrier to this because rather than acting as a measure of quality of the learning experience its focus is that of a customer satisfaction survey. Graham Gibbs has pointed out that there are much better ways of examining higher education from the perspective of quality in the excellent Dimensions of Quality, published by the Higher Education Academy.
But, all those benefits will remain unrealised unless the process can be implemented, and the process represents a fundamental shift in practice for many academics in many institutions, which of course, is the main obstacle. For the implementation to succeed it needs to reach a threshold. Implementing feedback mark two in one or two modules will probably fail to appreciably improve student achievement because it will be seen as a ‘one-off’, something new and novel and will probably hit resistance (from staff and students) because of that. It’s a little like immunisation in a population – you need a certain proportion immunised in order for everyone to benefit. Once implementation gets over that minimum threshold then it simply becomes the ‘way we do things here’, and the skills and benefits learned from the process in one module can be transferred to other modules. The problem is therefore one of institutional inertia. If the change required to get that level of benefit is so large, how are we ever going to achieve it?