Knowledge, Power and the Research Excellence Framework

Far beyond the arcane reaches of the Basel capital requirements, touched on in my last post, lies the swampy territory of the Research Excellence Framework, its very name bringing a whiff of fascistic thought control and striking terror into the heart of the nation's academics. Measuring research excellence is the sort of activity that creates an appealing soundbite for politicians of the ilk of Michael Gove but rapidly evades the mind, escaping into a realm of subjectivity and confusion, once it becomes an actual policy.

Thus have we, who labour in the mines of knowledge with sweated brow, attempted to resist the suggestion that there is anything meaningful about assessing a certain research paper as be 'worth' 3.25 or 2.75 according to criteria such as rigour, originality and significance. Sadly, it is often the papers that are particularly heavy on meaningless theory, or that are of the politically unchallenging variety that should find their way into the Journal of Overnight Discoveries, that score highest in these exercises.

The fact that the research evaluation system works against radical work has been a longstanding prejudice held by those of us who believe that quality cannot be enumerated and is unlikely to be clearly apparent until 50 years after a paper has been published, but at last we have some evidence to support our case. In the field of economics Fred Lee, professor of economics at the University of Missouri, has analysed the way in which economics papers are ranked and reached some disturbing conclusions. He has unearthed a self-referential and self-reproducing elite of economists who review each others' work, publish each others' work, and largely agree in terms of their conclusions about how the economy works.

As Lee correctly points out, this sort of groupthink amongst economists is a prime explanation for the disastrous nature of the financial crisis in 2007/8 and the fact that it was 'not foreseen'. But these charmed circles also work to control research funding, since work that is published in the most highly ranked journals, those with a 4* rating (yes, really) will ensure that their authors receive three times more research funding than those that only receive a 3* rating. Since the Cambridge Journal of Economics, the highest ranking economics journal that includes heterodox work and the site of Lee's paper, only scores a 3, heterodox economists are limited terms of the time they can spend writing and publishing and are unlikely to obtain jobs in the highest ranked universities.

Lee and his co-authors reach some startling conclusions, arguing that the control of the economics discipline is no less than an attempt to eliminate alternative views: 'the going concern model of UKeconomics puts into focus what others have identified as cultural, political, language, or social genocide of the "other"'. When in this case that 'other' might be the suggestion that capitalism can never be sustainable or that a debt-based money system will inevitably lead to global injustice, it becomes evident that this is a debate that needs to find its way beyond the walls of the academy and into the public domain.
. Tweet

Elite Syncopations

For those not familiar with the working of universities this post may seem rather obscure, but in reality it explains one crucial mechanism used by neoclassical economists to ensure that the routes to publication and promotion are controlled by their own. It explains how orthodoxy maintains its hold on the economics profession.

This is a large claim made for a research paper presented at the recent Association for Heterodox Economics conference in Nottingham by Harry Bloch, from Curtin University in Perth, Australia. It traced how Australian economists were involved in the process of deciding the quality of different pieces of research produced by their peers. In Australia this process is transparent; in the UK it is not. Hence the focus on scholars on the other side of the world.

The academics were told to rank the journals they commonly read into four categories; it was suggested that they should have 5% in A* (Nobel Prize material), 15% in A, 30% in B and 50% in C. The first conclusion from Table 5 is that economists are very hard on themselves: they considered a full 60% of journals in the also-rans category, a much lower percentage than other disciplines.

But it also demonstrates the domination of the field by proponents of econometrics. The method of regression analysis is the fast route to promotion. A mathematical model is always ranked more highly in terms of esteem than a paper which relates to a real-world problem, such as the financial crisis for example. Theory is rated well, with applied economics – that area which deals with policy – less so. Meanwhile ‘other economics’, such as that which actually questions the status quo, receives the lowest rating, with no journals featuring in the A* category at all.

It is also important to note that these were not everyday academics but Professors and members of professional societies. These are the guardians of orthodoxy, and their own work is likely to be in the field of econometrics and economic theory that presently dominate. As the table shows, none of the journals that covered heterodox economics was included in the A* ranking, making it extremely difficult for those outside the mainstream to achieve research funding. The process of ranking specifically excluded heterodox economists, although they were identified as a specific group - the only group to be excluded.

The second table (Table 7) illustrates the effectiveness of the gatekeeping by the orthodox. It shows the domination of the field by conventional economics while the majority of economists with different views languish at the lower tiers.

Bloch concludes his article as follows:

'The processes employed in the 2010 round of research evaluation under Excellence in Research for Australia (ERA) did not provide a fair assessment of heterodox economics research in Australia. The rankings used as the indicator of the quality for journal articles were unbalanced in favour of economic theory and econometrics and against applied economics and other economics (which included heterodox economics).'

We can expect no better form the UK's Research Excellence Framework process.
. Tweet