The Culture of Student Feedback

Feedback is a big issue in education – how much students get, and how (or if) they use it. Feedback features in the annual National Student Survey, and no matter how good the institutional results it remains one area that is frequently marked down.
David Boud and Elizabeth Molloy published a recent paper  on rethinking models of feedback that I found quite interesting. First, I’ll explain a little about the paper itself and then my thoughts on it, particularly as it relates to institutional culture.
Boud and Molloy distinguish between two models of feedback: the traditional model of feedback, which they call feedback mark one, and their model, which they call feedback mark two. They talk about traditional models by comparing them to the process in biological and engineering systems, i.e. that ‘information’ acts on a system in order to change the output. In educational terms, this is feedback as ‘telling’, feedback as a behaviourist pedagogy, and assumes that not only is the information given sufficient to actually produce a change, but that it’s unambiguous and that students will actually make use of it.
Feedback mark two is more of a developmental process and less of an add-on extra to an assessment. The aim is to transition students to become ‘agents of their own change’ so that they seek out information for improvement themselves rather than merely respond to it. Students become aware of what quality performance and feedback look like through dialogue, and through this process students develop the capacity to monitor and evaluate their own learning. Assessment tasks are designed so that students are engaged over a period of time so the generation, processing and use of feedback happens over a number of cycles. In other words, it’s making the receipt of feedback and reflection on it, (and the development of skills to do that) explicit.
Essentially, three elements work together: learners (and what they bring to learning), the curriculum (and what that promotes), and what Boud and Molloy call the ‘learning milieu’, which is the interplay of staff, students and the learning environment. Feedback shifts from being an act of teachers to an act of students (with teacher support), from a process involving a single source to multiple sources (with a corresponding shift from an individual to a collaborative act), and from an isolated event to a designed sequence of events.

My thoughts

Feedback mark one isn’t sustainable. First, it absorbs a huge amount of time and resources, but its effectiveness is questionable in actually influencing student behaviour and improving student outcomes, since students may simply look at the mark and ignore the carefully considered comments. Secondly, if there is a dual emphasis on student improvement and on improving NSS scores then the problem is that students don’t recognise the feedback they receive as actually being feedback. Thirdly, mark one feedback isn’t fit for purpose since it doesn’t equip students for life post-graduation.
Feedback mark two is, to my mind, a desirable development. It’s better pedagogically because students actively use the feedback. It’s better for the students because they are developing (and using) skills that will help them in lifelong learning and in employment after graduation. It’s better for the institution because it reduces the workload of academics (allowing more time for research, professional development, or improving their teaching in other ways), and because it improves the quality and performance of their graduates. The NSS is actually a barrier to this because rather than acting as a measure of quality of the learning experience its focus is that of a customer satisfaction survey. Graham Gibbs has pointed out that there are much better ways of examining higher education from the perspective of quality in the excellent Dimensions of Quality, published by the Higher Education Academy.
But, all those benefits will remain unrealised unless the process can be implemented, and the process represents a fundamental shift in practice for many academics in many institutions, which of course, is the main obstacle. For the implementation to succeed it needs to reach a threshold. Implementing feedback mark two in one or two modules will probably fail to appreciably improve student achievement because it will be seen as a ‘one-off’, something new and novel and will probably hit resistance (from staff and students) because of that. It’s a little like immunisation in a population – you need a certain proportion immunised in order for everyone to benefit. Once implementation gets over that minimum threshold then it simply becomes the ‘way we do things here’, and the skills and benefits learned from the process in one module can be transferred to other modules. The problem is therefore one of institutional inertia. If the change required to get that level of benefit is so large, how are we ever going to achieve it?

Advertisements

MOOCs: a stroppy teenager?

What I want to do in this post is look at where the xMOOCs (from the likes of Coursera, Udacity and Edx) sit within the educational landscape, i.e. where do they sit conceptually? In some ways, they’re undergoing a form of educational puberty. Like many adolescents, they have undergone a period of rapid growth followed by an existential crisis, in which they attempt to find their place in the world: who and what they are. I’ll look at three potential identities: as distance education, as conventional higher education at a greater scale, and as Continuing Professional Development (CPD). XMOOCs are a category with a diversity of provision so I am generalising, albeit from a perspective of personal experience.

MOOCs as distance education

In terms of their parentage I’d argue that xMOOCs are closer to traditional distance education rather than as an incarnation of online learning, and they’re much closer to distance education than the constructivist MOOCs they share an acronym with. As I’ve argued in another post if they are distance education, then it’s distance education done badly.

If they are trying to be distance education then a major problem is that their pedagogy (and content) follows too closely that of the campus models. Good distance education is not the same education made available (with a few tweaks) at a distance, any more than a video of a single person reading a novel out loud would constitute a ‘film of the book’. Also, distance education does not have to be synonymous with online learning – distance education worked (and worked well) when the medium of transmission was postal. When content and pedagogy follow closely the on-campus version it fails to take into account the challenges and advantages of distance education. The structure of the courses lack support and scaffolding for learners in two ways. Firstly, suppose that the target students are similar to conventional higher education students. The materials fail to help those students that, while capable in other senses, lack well-developed independent learning skills, and crucially don’t have access to the tutor interventions to bring them back on target because of the large cohort size (the ‘massive’ element of the acronym). Secondly, if widening participation is a goal then students are failed twice: not only may the lack independent study skills, but they may lack more fundamental study skills, either because they have lain dormant for a significant period of time or because they were never acquired in the first place. It doesn’t have to be this way – the UK Open University is a sterling example of providing higher education to an inexperienced student cohort through distance learning.

While independent learning skills are important Guri-Rosenbilt (2009) pointed out that conventional distance education was never about self-directed learning as a goal in itself, despite the benefits and transferable skills that that produced, but that they emerged as a necessary by-product of the process as it was mediated by the technology of the time, i.e. largely print materials by post. Scott (1995) cited in the same paper argues that traditional students can also lack the ability to learn autonomously. My view is that although this is true it matters less in face-to-face or blended environments since tutors can support students in various ways while they develop those skills. The problem with MOOCs is that it is precisely this scaffolding and support that is largely absent while at the same time requiring those autonomous skills in order to succeed in the course.

MOOCs as scaled-up conventional education

This is the key selling point of MOOCs, at least as far as the hype goes, that elite education is made freely available to a mass of people that otherwise would not have access. While I don’t discount that MOOCs can have a role in higher education, I think that the ‘elite education for all’ argument has been vastly oversold. There are a number of problems. The social and teaching presence that is a key part of a conventional education is missing. Interaction through forums is patchy. Personally, when studying on a MOOC, time pressures usually mean that I’m concentrating on covering the course content and I only resort to the forums when I have a particular issue or misunderstanding that I can’t solve on my own. My learning preference is that I’ll struggle, google and persevere before asking for help. The combination of time pressure and personal preference means that I make little use of the forums for discussion. Others students find that the sheer volume of forum messages can overwhelm them and make them feel that they’re not keeping up, especially when the design of that particular MOOC has made discussion an integral part of the experience. And of course the volume of messages mean that the primary interaction is student to student and not tutor to student. What this means is that a key component of active, meaningful learning is missing – the ‘conversation’ in Laurillard’s terminology, the Socratic passing back and forth of concepts and arguments for examination and modification.

Another missing element is that of accreditation. If MOOCs are for personal learning (and surveys have shown that many people taking MOOCs already have higher education qualifications) then formal accreditation is not required, although students may want some way of demonstrating achievement to others less formally. But, if MOOCs are to be incorporated within conventional higher education provision, then the problem of accreditation becomes much more important. How can we verify the identity of the person taking the assessment? And how do we solve the problem of meaningfully assessing student work at scale, particularly when scale is sold as a defining factor of the MOOC experience and value? I don’t think that the peer marking and machine marking that I’ve experienced on MOOCs would provide the integrity and reproducibility of normal academic practice that would be demanded. Accreditation often only targets content, but employers want more than just core knowledge, for example team working, communication, problem solving and decision making, and these are not being addressed in the current learning designs of MOOC. How they could be effectively assessed at scale is, again, a major problem.

A third issue is that of openness. What is meant by openness in the context of a MOOC? Open access, open materials, open source platform? Even in their current incarnations, the xMOOCs tend towards open access only (MIT OpenCourseWare makes its materials available under a BY NC SA license), although Edx does make its platform available as open source. However, it’s difficult to see how the materials could be made open for a MOOC offered within conventional HE either because a viable business model couldn’t be found, or simply because giving away your product for free would not be a venture capitalists’ first choice for their business model. Open source software gets around this problem by providing the software for free but selling the support services, but the support services in MOOCs are the staff (academics and others) that run the course. What prospect for universal public education if it’s the teaching and not the content that becomes the profit stream?

MOOCs as CPD

xMOOCs, as they currently are, may prove to be a much better fit as continuing professional development (CPD). My definition of CPD is fairly loose, encompassing both the ‘cover the content’ type of compliance training as well as courses with a much greater developmental emphasis. For one thing, the cognitive-behaviourist pedagogy that typifies an xMOOC is much better suited to the lower levels of Bloom’s taxonomy that a lot of the compliance training targets. That’s not to say that xMOOCs can’t aim at higher levels of Bloom (or that some CPD is not specifically designed to achieve the higher levels), only that the natural home of cognitive-behaviourist pedagogy is towards the lower end of Blooms. Another advantage is that the courses are often run by professional bodies and fee-paying, and with relatively small cohorts. Some professions are required to show evidence of ongoing development as part of their normal professional practice and some jobs have compulsory training requirements (e.g. data protection training). A combination of press-ganged employees and fee-paying participants in a relatively small cohort offers an obvious revenue stream for commercial MOOC providers whilst still providing a good quality experience.

Final thoughts

Are MOOCs destined to be the second class option, the fast food of the educational world – convenient, but lacking in real nutrition, bland and done on the run, compared to an alternative of, say, conversation with friends over a long, leisurely meal in a restaurant? I have a concern that xMOOCs could be seen as being for those for whom a conventional HE experience is unaffordable. That has wider implications for inequality within a society, but is entirely understandable within the neo-liberal model that a market is the answer to everything, and that if you can’t afford the premium product, tough. The potential incursion of MOOCs into the educational space of higher education institutions could be seen as an industrialisation process (in the sense of mass production) that seeks to undermine the ethos of HE, but which fits well with the marketisation agenda in the UK. Perhaps it’s simply that the collaborative, constructivist ideal of HE is incompatible (economically) with the ‘massive’ element of MOOCs.

References:

Guri-Rosenblit, S. (2009). Distance Education in the digital Age: Common Misconceptions and Challenging Tasks. Journal of Distance Education, 23(2), 105–122.

Scott, P. (1995). The meanings of mass higher education. Buckingham: The Society of Research into Higher Education and The Open University Press.