Potholes and fallen trees – the road ahead

Earlier this year (May 2015) we had an election here in the UK so it seems appropriate to try and scout out what may lay ahead, based on the budget statement, the business secretary Sajid Javid’s speech on productivity, and Jo Johnson’s speech to universities UK.

While there are some aspects that I agree with there are others where their impact very much depends on the detail of its implementation. Overall, there is a lack of coherence and a number of contradictions, although few are as blatant as this example from the energy section of the productivity document:

I'm pretty sure one of these isn't going to get done

I’m pretty sure one of these isn’t going to get done

or the proposal to double the proportion of disadvantaged students entering university. That sounds great, until you realise this is the government that tripled tuition fees, removed Education Maintenance Allowance that supported 16-18 year olds in study prior to university, and is removing maintenance grants aimed at the poorest students only to replace them with more loans.

An overriding theme throughout all of these speeches and documents is that of monetary value being the only measure of importance. Johnson says that only half of students polled thought that their course had provided good value for money[1], but a huge amount hangs around how we choose to define value. Is it simply a business measure of return on investment – the improvement in our salaries for the cost incurred? Is it that our path to a ‘good’ job (another loaded word) is eased? Or is it the knowledge that we were exposed to, our lecturer’s enthusiasm for their specialism that inspired us to change direction? Or simply watching the sun come up with friends after the final summer ball with that bittersweet mix of relief at having succeeded and regret that our time is ending? Do we measure ‘value’ in the ache of intellectual exertion and growth or in the frame we hang on the wall afterwards? I remember my eldest daughter wanting help with her English essays (i.e. ‘tell me what to write’) and us having the same conversation a number of times that the homework was the research, analysis and synthesis of the raw material. The homework was the process and the piece of paper handed into the teacher was just the proof that you’d done it.

There is a direct link to earnings made[2] and the phrase ‘… enable education providers to assess their effectiveness in delivering positive labour market outcomes for their students’ appears so there is little doubt what definition of value is being used. It’s obvious no one actively involved in the process of education wrote this. Is this really how we want to gauge higher education, in “positive labour market outcomes”? Yes, we want our students to be successful in whatever field they choose, but that should naturally follow on from quality education anyway. Having those outcomes may be appropriate for vocational courses (of whatever level) where that’s an explicit objective of the course anyway. That’s not a dig at vocational education, by the way. We have a very biased attitude towards vocational education in the UK, seeing it as the poor relative of higher education and a refuge for those lacking the ability to do a degree. That’s profoundly wrong. We only have to look across the channel to Germany to see what a less prejudiced approach to vocational education can achieve for individuals and the society they are part of.

As part of the push towards transparency and ‘more informed choices’ elements of the US version of the National Student Survey (NSS) are to be piloted. This is good because the American survey is much more fit for purpose in terms of measuring what’s important for teaching and learning rather than the ‘customer satisfaction’ surveys we use in the UK, but it does depend on how those elements are integrated with the current NSS. Jo Johnson’s speech talks about using these elements to measure ‘learning gain’. My big fear here is that the metric we use is an easy-to-measure proxy rather than a true representation of the thing we’re interested in. The speech defines learning gain as ‘the improvement in knowledge, skills and work-readiness’. Leaving the issue of work-readiness to later, these are all graduate properties that are emergent and as such at this level will be relatively difficult to measure with reliability and validity. What this will mean, as has been the case with the ‘value-added’ methodology in schools, is a push towards a proxy that may be only loosely representative of the true quality we’re trying to measure.

Another element to this danger of proxy worship is the next paragraph in the speech which, while acknowledging that “independent learning is vital”, then states that universities should be more transparent about how student time is split between lectures, seminars and tutorials, as well as who is doing the teaching. This information is valuable, but is not the whole story. I’d certainly welcome information on who is doing the teaching being more available, even if only to avoid the explosion in the use of adjuncts that has happened in the USA. Contact hours, however spent, will vary according to discipline, with STEM subjects typically having higher contact hours than the humanities, and again I can see no issue with this information being available in principle. The issue is that without some more qualitative metric to counterbalance them these numerical measures will distort how higher education is provided. Independent study will be sidelined in favour of contact hours since that will be the measure that importance is attached to, but how are those contact hours to be delivered, and by whom? Staff time (already stretched) costs money and student numbers are increasing, so if there are to be no additional resources provided and we can’t (and shouldn’t) reduce costs by using adjuncts or post-graduates to do the teaching, how on earth is it all expected to come together in some sort of coherent whole?

There is a worrying mention of the use of competition to drive up standards. It’s worrying in two ways. First, the practice of education is a collegiate activity and we succeed, as Newton put it, by “standing on the shoulders of giants”. Secondly, the productivity plan[3] makes clear what form this will take place by saying “widening the range of high quality higher education providers” and “remov[ing] barriers preventing alternative providers from entering and growing in the market”. To me, this sounds very much like fast-tracking a large-scale expansion of the for-profit sector in UK higher education. Given the current high standing with which UK higher education is held and widespread concerns over some of the for-profit colleges in the United States this seems a very risky strategy to pursue.

The productivity plan then goes on to say the government “will explore options to allow the best providers to offer degrees independently of existing institutions before they obtain degree awarding powers”[4]. I’m a little confused by the inclusion of this sentence. Courses can be already taken at institutions that don’t have their own degree-awarding powers provided that they are accredited by an institution that does – my own post-graduate teaching qualification was taken at an further education college but validated by a university – so why the need to ‘explore options’? Also, how is ‘best’ to be measured, and by whom?

The Jo Johnson speech mentions ‘innovation in terms of course length and design’[5], with an explicit mention of degree apprenticeships. This is a welcome acknowledgement of the importance of vocational education. Unfortunately, Johnson also mentions two-year courses on the grounds that they offer a more accessible route to higher education and, together with degree apprenticeships, offer a ‘faster route to productive employment’. I think it’s much more likely that accessibility to higher education is more adversely affected by the level of debt students will have on graduation rather than the length of the course. And why should a faster route to employment even be a goal? What evidence is there that a shorter course would produce graduates with comparable skills and attributes to those that graduate after three years?

Two-year courses would be attractive to those new for-profit providers since they can produce more graduates than a conventional university in the same time, particularly if the Teaching Excellence Framework allows some institutions to self-certify their level of performance (as has been done in schools where those that received the higher grades on inspection were re-inspected on a longer timescale and effectively inspected themselves in the intervening period). Together with the uncertainty around the degree-awarding powers for new providers these ‘more accessible’ pathways to HE and shorter routes to employment seem to be a recipe for dumbing down, slipping standards, and increasing ‘shareholder value’.

Johnson also talks about course choice and curriculum design[6], saying that not enough is being done to ensure graduates are prepared for work, and quotes a CBI/Pearson Education and Skills survey that nearly half of employers (47%) felt that universities weren’t doing enough to make graduates job-ready. Well, that would depend on how you chose to define ‘prepared’ and ‘job-ready’. My own response to that 47% of employers is: do your own training. We’re not your HR department. Graduates should have graduate-level, transferable skills – that’s the contribution from higher education to being job-ready. That means graduates can adapt to the specifics of a particular role or industry, but it’s the employer’s job to mediate and support that adaptation. If business has any evidence that we’re not producing that standard of graduate (beyond vague complaints that graduates are not ‘job-ready’) then they should produce it.

Johnson then says that businesses should not just be customers of universities, but “active partners”, and says that business outreach should be a core function with influence over curriculum design. Absolutely not – I repeat, universities are not part of a business’ HR department. Education is not the same as job training. Should businesses have some involvement in curriculum design? Yes, of course – collaborations with industry can be very productive for both sides – but decisions around the curriculum should rest with the university except in rare cases.

I’m not anti-business – trade and enterprise have always been a part of human societies and business is integral to modern societies – but I am against the fetishistic worship of business and entrepreneurs as having all answers to all questions. Increasingly, mainstream economics and politics seem to revolve around the mantra ‘the answer is a free-market. Now, what was the question?’ although there are others such as Naomi Klein and Paul Mason putting forward alternative viewpoints. Education has huge societal value beyond servicing the needs of business, but this seems to be increasingly neglected. A telling point is that higher education in the UK is administered not from the Department of Education, but from the Department of Business, Innovation and Skills.

Overall, the direction of travel for higher education in the UK does not look good. Statements and aims often explicitly contradict each other, such as wanting to widen access while simultaneously constructing more disincentives. The language of business dominates with the focus and metrics primarily targeted towards financial measures rather than anything innate in the process and practice of education. These are not the characteristics of a healthy higher education system, unless the function of that system is to be a subsidiary of business. These are not the characteristics of a system that Johnson himself acknowledges is a “world leader in higher education … and as number one in the world for the impact of our research”.

footnotes

1Johnson – Meeting value for money expectations.
2Johnson – More informed choices.
3Productivity plan – section 4.10
4Productivity plan – sections 4.9 and 4.10
5Johnson – Widening Participation.
6Johnson – Course choice and curriculum design.

Advertisements

The Culture of Student Feedback

Feedback is a big issue in education – how much students get, and how (or if) they use it. Feedback features in the annual National Student Survey, and no matter how good the institutional results it remains one area that is frequently marked down.
David Boud and Elizabeth Molloy published a recent paper  on rethinking models of feedback that I found quite interesting. First, I’ll explain a little about the paper itself and then my thoughts on it, particularly as it relates to institutional culture.
Boud and Molloy distinguish between two models of feedback: the traditional model of feedback, which they call feedback mark one, and their model, which they call feedback mark two. They talk about traditional models by comparing them to the process in biological and engineering systems, i.e. that ‘information’ acts on a system in order to change the output. In educational terms, this is feedback as ‘telling’, feedback as a behaviourist pedagogy, and assumes that not only is the information given sufficient to actually produce a change, but that it’s unambiguous and that students will actually make use of it.
Feedback mark two is more of a developmental process and less of an add-on extra to an assessment. The aim is to transition students to become ‘agents of their own change’ so that they seek out information for improvement themselves rather than merely respond to it. Students become aware of what quality performance and feedback look like through dialogue, and through this process students develop the capacity to monitor and evaluate their own learning. Assessment tasks are designed so that students are engaged over a period of time so the generation, processing and use of feedback happens over a number of cycles. In other words, it’s making the receipt of feedback and reflection on it, (and the development of skills to do that) explicit.
Essentially, three elements work together: learners (and what they bring to learning), the curriculum (and what that promotes), and what Boud and Molloy call the ‘learning milieu’, which is the interplay of staff, students and the learning environment. Feedback shifts from being an act of teachers to an act of students (with teacher support), from a process involving a single source to multiple sources (with a corresponding shift from an individual to a collaborative act), and from an isolated event to a designed sequence of events.

My thoughts

Feedback mark one isn’t sustainable. First, it absorbs a huge amount of time and resources, but its effectiveness is questionable in actually influencing student behaviour and improving student outcomes, since students may simply look at the mark and ignore the carefully considered comments. Secondly, if there is a dual emphasis on student improvement and on improving NSS scores then the problem is that students don’t recognise the feedback they receive as actually being feedback. Thirdly, mark one feedback isn’t fit for purpose since it doesn’t equip students for life post-graduation.
Feedback mark two is, to my mind, a desirable development. It’s better pedagogically because students actively use the feedback. It’s better for the students because they are developing (and using) skills that will help them in lifelong learning and in employment after graduation. It’s better for the institution because it reduces the workload of academics (allowing more time for research, professional development, or improving their teaching in other ways), and because it improves the quality and performance of their graduates. The NSS is actually a barrier to this because rather than acting as a measure of quality of the learning experience its focus is that of a customer satisfaction survey. Graham Gibbs has pointed out that there are much better ways of examining higher education from the perspective of quality in the excellent Dimensions of Quality, published by the Higher Education Academy.
But, all those benefits will remain unrealised unless the process can be implemented, and the process represents a fundamental shift in practice for many academics in many institutions, which of course, is the main obstacle. For the implementation to succeed it needs to reach a threshold. Implementing feedback mark two in one or two modules will probably fail to appreciably improve student achievement because it will be seen as a ‘one-off’, something new and novel and will probably hit resistance (from staff and students) because of that. It’s a little like immunisation in a population – you need a certain proportion immunised in order for everyone to benefit. Once implementation gets over that minimum threshold then it simply becomes the ‘way we do things here’, and the skills and benefits learned from the process in one module can be transferred to other modules. The problem is therefore one of institutional inertia. If the change required to get that level of benefit is so large, how are we ever going to achieve it?