Thoughts on the UK’s Teaching Excellence Framework
On 22nd of June 2017, the UK’s Teaching Excellence Framework (TEF) results were released. While this immediately led to celebratory noises from the various ‘winners’, the prominent ‘losers’ tended to call into question the validity of the ranking as unreflective of actual teaching quality. I want to talk about this response and the context that surrounds TEF, and how my own thinking about it (and metrics in general) differs somewhat from this ‘accurate vs. inaccurate’ characterisation of TEF.
Firstly, TEF is of course complex. Wonkhe has a good summary of the TEF mechanism, but I’ll give a short version here. TEF is a UK government-run assessment scheme for rating the ‘quality’ of ‘teaching’ (I use both words advisedly). It requires UK universities submit justifying evidence alongside various metrics (e.g., National Student Survey results, graduate employment data, etc.). The TEF process then sorts participating universities into ‘Gold’, ‘Silver’ or ‘Bronze’ categories. These categories are then tied to fees that universities may charge students (although this mechanism is not in place yet). TEF is also conceptually connected to the Research Excellence Framework (REF), which is the UK’s national evaluation of research output conducted every 4–5 years (last time 2014). REF, being older, is far more involved than TEF, relying on a significant number of ‘peer review’ panels (using that term advisedly too) to inspect selected research output from all UK institutions in order to then rank them (which is then tied to core funding distribution).
Three points to note first:
1. I don’t think my observations below are particularly original or surprising.
2. I am no expert in politics or the history of government relationship to universities or in university governance.
3. It should go without saying but quite obviously I do indeed think universities should strive to be excellent in teaching. Just not in this way.
Trend and ideology
TEF is the latest way that successive governments in the UK have sought to introduce (or impose, depending on your point of view) market dynamics on the university sector. Broadly this reflects the advance of something that looks a bit like scientific management. I believe TEF is traceable to longer term trends of ‘new public management’ and managerialism that emerged in the 1970s and 1980s to reshape the public sector, particularly through the application of metricisation, performance monitoring, and introduction of competition mechanisms. Even more broadly, if could be said that TEF is reflective of the logic of neoliberalism that has been adopted (or at least tolerated) by all recent UK governments as a model for most shaping sectors of society: i.e., privatisation, financialisation, deregulation, etc. I say this more as a matter of fact than a suggestion of right or wrong. The core question is the relevance of that kind of model to higher education and universities as institutions.
The REF informed TEF; in fact Michael Barber says as much in his recent speech on the Office for Students. But the conditions for TEF’s appearance have been steadily establishing themselves for a long time. The older version of the REF, the Research Assessment Exercise (RAE), started in the 1980s as the application of public management ideas gained traction. The introduction of university fees for students at the end of the 1990s accelerated this (it was a solution to widening participation and access). All of these incremental moves lead us towards the reconceptualisation of universities as corporations, students as customers, and the introduction of business processes to university management.
In a way this is key to understanding the logic of TEF and the form it takes, i.e., reliance on metrics. Some kind of metricisation is a key requirement for the introduction of market dynamics in that it provides the ‘signal’ for market mechanisms to operate. (Note that just because the Gold-Silver-Bronze categorisation system is adjectival does not mean it is no longer functioning as a metric.)
As I mentioned, the ideology of TEF seems to be based in the assumption that market dynamics are the way to improve almost anything. Since this is the essential model to be applied to all sectors of life, there clearly can be no suspension for universities.
This leads me to two big questions. Firstly, the sheer complexity and cost of TEF and REF is enormous. The desire is for TEF to become subject specific, thus incurring even more administrative burden. It is interesting to see that a cost-benefit argument is specifically not used with either TEF or REF in order to justify the expenditure. More on that in the next section.
The second big question here is about what I guess you could call the epistemology of TEF’s instigators. It needs examining. Do they believe metricisation throws into the light some hidden naturally occurring order of things embedded into universities but hitherto invisible? Or do they view it metricisation as a mechanic to manipulate and control the university sector? Again, there is very little forthcoming on this, perhaps because even suggesting such questions threatens to shed some light on the central assumptions of TEF.
Given the trends I think it possible (but not necessarily likely) that current and future governments will gradually attempt to move TEF towards an Ofsted-like approach to assessing higher education, i.e., greater oversight, greater powers, involving in-depth inspections (preferably by peers but there a few guarantees), and stronger sanctions. The use of ‘Teaching Excellence’ terminology rather than ‘Student Experience’ implies that indirect measures of teaching quality are only the beginning.
This leads to the following points on language and strategy.
Language and strategy
The language of TEF and indeed REF is positioned in ways that make it seem churlish to oppose. I suspect this is intentional, but I don’t know. The strategic move here has been to align TEF (aspirationally) with teaching rather than ‘reported student experience’ (which would be more accurate). But in line with what I noted above, this is a neat trick since TEF currently only uses indirect measures of teaching in order to map out institutions to the Gold-Silver-Bronze categorisation scheme — in other words there is no direct examination of teaching at all (e.g., via observation).
TEF is thus explicitly affiliated with categories of ‘teaching excellence’ and the rigour-inflected notion of ‘framework’. This means opposition to TEF then tends to affiliate with inversions of those things, because that’s how language generally works. No-one can be reasonably be against ‘teaching excellence’, therefore criticising the very idea of the TEF is met with astonishment (compare this with criticisms that merely seek to tweak its metrics to be ‘more accurate’ or ‘representative’ — i.e., that accept the very idea). To critique TEF in this fundamental way — essentially a critique of ways of knowing, i.e., epistemological — is thus readily taken to be against ‘teaching excellence’ as important (which everyone wants) and therefore ‘you are bad’. It would be good to see the discourse advance a bit here.
Institutionalisation
It also seems that, with the TEF and the REF, there is a gradual process of institutionalisation taking place for academics and those working in universities. Surely we should expect academics, as critically-minded people, to be very careful when engaging in metricisation of, say, natural or social phenomena? (It’s their job!)
It goes like this: At first there is outright skepticism. This then crumbles into skeptical participation (with a nudge and a wink indicating ‘what we all really think about it’). But when the prizes are awarded, the process of acclimatisation is complete, particularly for the winners.
Gold-award institutions in particular come to feel that the metric has been truthful and proven what they already knew: that their teaching is indeed excellent. While schemes such as TEF ‘have their teething problems’ for some there no longer is much skepticism about the fundamentals of the exercise. The role of self-congratulation is important here. As Emilie Murphy states “by applauding TEF results we implicitly accept this framework and its methodologies”.
As the acclimatisation proceeds we see an institutionalisation process taking place. Internal, institutional systems are set up to replicate the TEF or REF in order to ensure that in the next round we are favoured / improve / whatever. After this, internal-TEFs or internal-REFs are then connected with performance assessment, progression, and promotion as part of the logic of managerialism. The end result is that externally-specified and mandated metricisation mechanisms come to significantly influence and shape the guts of academic life: how papers are written and published, which research ideas are pursued, how teaching is configured, how students are treated, etc. This is often glossed as ‘game playing’ but this overlooks the wider potentially deleterious effects that happen downstream as a result of that game playing.
Of course, it is unfair to paint this picture of acquiescence without noting that schemes like TEF (eventually) and REF (currently) wield huge sticks in order to get that level of compliance. If it was me in charge I accept I’d probably have to acquiesce too. For TEF the stick is student fee levels, while for REF it is core funding. The stakes are enormous for universities: and you have little choice but to play. Rather than blaming the people involved in the acquiescence or institutionalisation, perhaps it’s more like the frog-in-boiling-water parable: it’s the slow creep of accepting the system in a piecemeal way which is the problem.
The really sad thing about TEF and institutionalisation is that universities are already saturated with ways of assessing and improving teaching. Student evaluations of modules, surveys, peer observation and feedback, connecting PGCHE certification to promotion requirements, meetings and workshops on sharing best practice, institutional support (e.g., courses) for developing teaching skills and techniques, etc. There are many obvious ways to improve teaching from a national point of view that don’t involve logic of metricisation. While I’d like to be pleasantly surprised, I do doubt whether the TEF is really going to genuinely enhance teaching.
Originally published at notesonresearch.tumblr.com.
