Earlier this year, HEFCE and its sister funding councils announced a new policy for Open Access (OA) in relation to the next Research Excellence Framework (REF). The REF is a periodic exercise to assess the research undertaken by UK HE institutions, assessing the quality of research outputs (such as publications, performances, and software), the impact of this research on wider society, and the health of the research environment in institutions within which research takes place. Scores are given for each ‘unit of assessment’ (disciplinary area) submitted by an institution, and these scores drive research funding for that institution for the following years.
A lot of money flows to institutions as a result of this exercise. The big number is the £1.6bn per year that HEFCE allocates in quality-related research (QR) funding to English institutions, and further funding is provided to institutions in Scotland, Wales and Northern Ireland by their respective national HE funding bodies. The REF scores are also used to calculate how several hundred million pounds in research capital grants are allocated to institutions in England, and the scores are used by institutions and others for comparative analysis in league tables. It is safe to say that this exercise is a very big deal for institutions in the UK.
The policy for OA was introduced after an intensive, unique, two-staged, UK-wide consultation exercise, undertaken over a period of roughly 18 months. The consultation firstly sought views on how the REF might introduce a requirement that outputs submitted to the next REF (expected towards the end of this decade) be published or made available as OA. The overwhelming view was expressed that it would be inappropriate, as well as financially unviable, for the REF to expect authors and their institutions to make outputs available via the ‘gold’ route. The funding councils instead consulted on and implemented a policy that is much ‘greener’ in nature: a requirement that journal articles and conference papers be deposited in an institutional or subject repository and made available for read/download after any embargo had elapsed.
The academic and HEI community, being all too aware of the unresolved issues with gold OA, seemingly breathed a sigh of relief at that point. But the world has moved on somewhat since then. The Institute of Physics Publishing and the Royal Society of Chemistry have introduced mechanisms to deal with the costly issue of ‘double-dipping’ by offsetting institutions’ subscription payments by the amount paid in gold OA fees, and it is understood that other major publishers will be following suit soon. More recently, Nature Publishing Group converted its flagship OA journal Nature Communications into a fully OA title, and the Royal Society launched a new OA journal, Open Science. It is not just publishers that are driving progress towards: just a couple of weeks ago the William and Melinda Gates Foundation announced a very strict OA policy for its funded authors to follow: by 2017, all papers arising from Gates funding must be made available immediately on publication as OA with a CC BY licence. The OA world seemingly erupted in praise for Gates, the policy being feted as a victory for openness in research.
The purpose of this post is to re-examine the REF requirements in light of these developments. The REF policy encourages fast and free OA, but its minimum requirements allow restrictive copyright and long embargoes to be placed on research publications. Neither of these, in my view, can be considered to deliver true OA. And one can’t help noticing that some of those praising Gates for pushing things forward have also criticised the REF for not going far enough in freeing up research. Leaving aside the obvious point that the REF requirements will make the vast majority of UK research publications available for free in some form, which is a huge improvement on today’s situation and widely seen as a ‘game-changer’, it is worth revisiting whether the REF rules could, in fact, be tightened to require the immediate and liberal use and reuse of research publications as a condition of eligibility for assessment.
Immediate access
The REF rules say that papers must be made available in a repository within 12 months of publication (24 months in the humanities and social sciences). This was a pragmatic decision: most publishers allow green OA, but often say that papers can only be made OA after an embargo period has passed. As it happens, the figures of 12 and 24 months allow 96% of papers to simultaneously meet the REF requirements and publishers’ requirements, and given only 12% of papers are currently being deposited in repositories the gains in OA achievable within these parameters were felt to be more than sufficient as a substantial first step.
Nonetheless, if we wished to shorten these delays, we’d have a number of options. The first would be to tighten up the requirements. HEFCE analysts have shown that if the embargo maxima was shortened to 6 months (12 months in HSS), and publishers did not change their policies, the proportion of papers that could comply with the REF policy would fall from 96% to 67%. If one went further, and required immediate access to papers, the proportion of compliant papers would drop to around 40%. This raises an important policy question – what to do about that huge chunk of research papers that wouldn’t comply with stricter rules? The current policy is fairly lenient in how it handles those few papers that can’t meet the requirements – a sensible and pragmatic approach, if one accepts the principle that if one can achieve 96% OA without fussing over embargoes then one should probably not worry too much about that remaining 4%. But is that a feasible approach for 60% of research papers? And how lenient should one be?
If one were very strict, and said that only papers that were available within those embargo periods were admissible to the REF – perhaps saying that any papers that didn’t meet the requirements would have to be justified on a case-by-case basis with a very high rate of rejection – one is effectively barring publication in those journals. It is easy to see how upsetting that would be for academics and their institutions. Some excellent papers would be excluded from the REF. Some papers would be published, academically speaking, in the ‘wrong’ place. Whole disciplinary areas might be left with no compliant publishing options. Authors and their institutions might feel compelled to pay article processing charges (APCs) to comply with the REF. Yes, some publishers would shift their rules in order to allow UK authors to submit and publish in their journals. But how many would do this? Indeed, how many could do this? (Not to mention the risk that some publishers would see this as an opportunity to ‘force’ authors down the gold route…)
One might decide to be tough, stick to one’s guns, and simply accept those risks. I wonder how much concern this would cause at very senior levels within institutions that have significant lobbying power. What would the government think about a UK research assessment that barred academics from publishing in ‘top’ journals? What effect would this policy have on inward mobility if overseas researchers saw the UK as a place where you couldn’t publish in certain journals? Would that stick for long?
It is possible to see how one could mitigate these risks, of course, by doing one of two things. The first would be to relax the approach to exceptions, effectively letting in most (or nearly all) of the non-compliant papers. This, in my view, would be totally pointless. All it would do is increase the amount of work that institutions would need to do to consider and record exceptions in their REF submissions. One may as well start from the position of having relaxed criteria in the first place, and save on workload.
The second is to increase the emphasis on the gold route to OA. Gold OA has no embargoes, of course, because the risks that the embargos are designed to protect against are underwritten via other means (commonly by the payment of article processing charges). Papers published via the gold route can be deposited straightaway. Still thinking about compliance, then, a key question is how many of the papers being published today are published in venues that have a gold OA option. A good number of that ‘compliant’ 40% figure, I’d wager. But certainly not all of the remaining 60%. Journals like Nature offer no gold option. One would need to have a way of dealing with those papers that are published in venues (like Nature) that do not allow green OA with a zero embargo and do not have a gold OA option. It would be interesting to do some analysis to see how many papers we’d be talking about here.
Liberal reuse
Before getting into questions of cost, I want to consider the implications of tightening up the licensing requirements. At the moment, the REF policy is pretty permissive about licensing. The requirement is that outputs must allow anyone with an internet connection to discover, read, download and perform in-text search on them. This means that papers in repositories are OK, but those proprietary platforms such as ReadCube that forbid people from downloading the paper are not good enough. Plugins that embed papers in repositories (in the manner of a YouTube video on Facebook) need to allow downloading to comply. But the requirements do not stretch beyond that. Papers do not have to be made available under a Creative Commons license to comply with the policy, and if they were, a restrictive CC BY-NC-ND licence would be sufficient.
This is a far cry from the very permissive CC BY licence that Gates Foundation is requiring for all the published papers arising from research it funds. Why is the REF policy so relaxed about licensing? Some of reasons are similar to those given above for embargoes: ease of compliance, and the judgement that the policy requirements in their current form constitute a big enough step in the right direction. While it is very common for publishers to allow authors to post copies of their manuscript in repositories, it is rare for them to allow any special licence to be attached to this posting. It is not the case that publishers explicitly forbid CC licences; they do not need to. They own the copyright, and by saying nothing they automatically assert all their rights.
Will toll-access publishers allow papers to be made available under more permissive terms that those set out in the REF policy? I think they will have difficulty with this happening via a green OA route. Green OA is acceptable to publishers at the moment for a variety of reasons. The main reason is that the academic community has long insisted that publishers allow them to post their paper on their website or in a repository; without this, I don’t believe green OA would even exist. Beyond this, I believe publishers tolerate green OA because they can do all sorts of things to reduce the value of the green OA deposit, effectively driving value towards scarcity (the toll-access paper). Embargoes are one such mechanism. Special conditions on green deposits – such as the requirement that the deposited file not be the final printed version of record or the requirement that the repository version link directly to the toll-access version – are others.
Is copyright such a mechanism? Technically speaking, copyright on green OA papers does not drive value towards the more scarce offering (the toll-access version of record). However, less than full copyright on green OA papers might drive value in the other direction – away from toll-access papers and towards green deposits – because the OA version is more useful. As long as toll-access publishers continue to exploit copyright in the way they do now, they will be concerned to protect the copyright of their work from the ‘threat’ of more liberal licensing. So no, I don’t believe publishers will be keen to allow green OA deposits to be made available under more liberal terms. That doesn’t mean they won’t do it in response to a REF policy, or RCUK policy for that matter, and in any case there is clear merit in engaging in discussions about what liberal licensing of green OA looks like. It simply means that they won’t make these moves without some discomfort. And, as with shortening or eliminating embargoes, some will be perfectly happy to do it, others will simply refuse to do it, and some will see an opportunity to force payment of gold OA fees.
Again, one could either accept those risks and press ahead regardless with a strict requirement for CC BY or some other licence. This has similar consequences to enforcing a stricter policy on embargo periods: it either restricts author choice, or it restricts the ability of the REF to accept the ‘best’ research (assuming that ‘best’ does not include a judgement of openness, which is not inarguable). Or it forces payment of gold OA fees. Would we want our national research assessment system to be bound by such restrictions? Even if we did, would senior academics, politicians, institutional managers, sector leaders, learned societies, publishers and overseas academics accept it without a fight?
This leads us to the inevitable conclusion that if one wants to move towards a stricter policy around immediacy and permissiveness of access to publications submitted to the REF, one must be willing to accept that stricter conditions attached to green OA will not work on their own. The green OA environment means that any stricter conditions on OA would need to be more leniently enforced, because they could not be universally adhered to. Lenient enforcement in the context of the REF is a very difficult thing to achieve because institutions tend to worry about how strictly the parameters of any lenient approach get enforced; stricter and more rigid enforcement of more lenient rules works much better. (Lenient enforcement also creates the problem that it relaxes the pressure on publishers to adhere to the stricter rules, making any change – including incremental change – much less likely.)
Moving beyond green OA
So if we can’t achieve faster and more permissive access through green OA alone, we must accept that gold OA will be a necessary part of how authors and institutions can comply with stricter policies. To illustrate, if it were to emulate the Gates policy, a revised REF policy would state that, to be eligible for the REF, publications would have to be available under CC BY licence immediately after publication. Compliance would happen via either the green route or the gold route.
A policy this strict would drive a significant amount of business to the gold OA model. This raises some significant questions, which I’ve started to tackle below. (Disclaimer: the numbers here are taken from a variety of sources and rely on a large number of assumptions that are not factored in. I have given these calculations for illustrative purposes only.)
- How much would this cost? We know that the REF policy will have a much wider effect than just on those publications submitted to the exercise; authors and institutions cannot know which publications they wish to submit in advance of the REF submission phase, so any policy will drive OA to the full range of research publications happening in UK HEIs. Currently, the UK produces around 140,000 papers per year. Not all of these will be produced an author or authors within UK HE, but the vast majority are, and the volume of published research increases annually by significantly large proportions that 140,000 is useful enough as a ball-park figure for our purposes. If we accept the above reasoning that the green route is pretty much useless for delivering immediate OA with a CC BY licence, then we should conclude that most, if not all, of those 140,000 papers will need to be channelled through the gold route each year. How much will this cost? This is very tricky to predict, but if one takes the Finch middle-ground prediction of £1,750 for the mean APC, and assumes that publishers either already have or will shortly introduce a CC BY gold OA option, the back-of-the-fag-packet UK gross APC requirement would come to £245m per year.
- How would this be paid for? Well, not all of that £245m would need to be found by institutions and authors: RCUK and the Wellcome Trust, for instance, would already be covering a proportion of this sum from their grants (£24m on recent figures). The remainder requirement is therefore roughly £221m. If this were a REF policy, then it would be fair to say that the funds that were allocated on the back of the REF should be used to cover this sum. HEFCE paid out £1,558m in quality-related (QR) research funding in 2014-15. Assuming that English universities and colleges account for 81% of the total UK output (based on UK funding body contributions to joint projects), the APC requirement would involve taking a £179m chunk out of QR (roughly 11% of the total), allocating this toward APCs. (These numbers are extremely sensitive to fluctuations in research output, APC prices, publication patterns, the OA policies of other research funders, and overall research funding levels, many of which would appear to put upward pressure on this 11% figure.)
- What effect on research performance? If the funding model did not change to reflect this new requirement, it would create funding problems for research-intensive institutions that had lower REF scores. The APC funding would therefore more sensibly be awarded by volume of research published alone, meaning that the total HEFCE research allocations (QR+APC) would be less concentrated towards excellence. Taking a ratio for annual return on investment for R&D of between 0.2 to 0.5, the loss in research performance in the first year alone would amount to between £36m and £90m (not considering any de-concentration effects); this number effectively doubles each year as R&D impact is long-term and cumulative. For comparison, the positive long-term NPV from OA has been very hard to quantify in recent years; Finch even calculates the figure as a negative number, recognising that NPV calculations will miss many positive externalities of OA. But ut it is reasonable to assume that in the short to medium term the additional gains from pure gold OA are bound to be negligible as subscription payments will need to continue for the other 94% of global research publications. Simply put, in a 100% gold REF world, we’d do 11% less research to make just 6% of the world’s publications OA, which is damaging in itself, and as long as we’re going it alone the UK taxpayer gets a bad deal from this sort of OA.
A few caveats here. In my second bullet point above, I assume that QR funding must be allocated away from research costs and towards APCs. I have made two further assumptions in connection to this. The first is that additional funding for APCs will not be available from government or elsewhere. (This seems like a reasonable assumption, based on recent central funding decisions in this area.) The second is that all QR goes directly to supporting research activity. This is less realistic: it is clear to me that some QR funding is currently used by institutions to cover journal subscriptions and other indirect costs. I generally agree with Finch that the end point of gold OA is widespread subscription cancellations; if we get to that point, the loss in UK research performance from a global 100% APC-driven gold OA outcome could be much lower, if not quite zero, because subscription money could in theory be transferred back into research budgets. (I say not quite zero because publishers freely admit that the price they’ll set for an OA system will be much higher than that they currently set for subscriptions.)
In sum, if we were to fulfil the aims of OA purists and design a REF policy that looks like the one Bill Gates has just announced, we have to accept that gold OA would have a significant role to play – indeed, gold OA would probably account for close to 100% of research papers published by UK HEIs each year. However, it’s obvious that the short-term costs of simultaneously paying APCs and subscriptions would require dedicated additional support of the sort I have described above, because they would otherwise be unaffordable. While the benefits of OA itself are substantial in the longer run through increased knowledge stock, positive externalities and the magical but hard-to-predict specific effects of openness interacting with opportunity, the marginal benefits of immediate CC BY OA compared with embargoed copyrighted OA are less obviously delineated. If we were to go down this road without measures in place to control the short and medium term costs (such as those introduced by IoP and RSC), the UK would take a substantial hit on research output and performance in the short to medium term, and possibly in perpetuity.
Postscript: this post ignores (a) the controversies in certain communities around the CC BY licence, and (b) the commonly heard prediction that universal green OA will somehow deliver a sustainable gold OA future all on its own.
Update: As noted by Cameron and Graham in the comments below, my analysis of the costs of OA is not as sensitive as it could be. In my defence, that wasn’t really the point of it, and hopefully I’ve given enough disclaimers to show that this stuff is pretty difficult to do well. Cameron points to some work done by PLOS that reveals that only about 40-50% of affiliated papers get billed to a given country or institution. This means that not all of those 140,000 papers will be billed to the UK. Cameron himself notes that any country pursuing the fully gold route, as I’ve set out, would see its proportion rise, as co-authors in other countries would be less likely to step up to the plate. Insofar as we’re talking about transitional costs of ‘going it alone’, this is a very real risk in our case. However, this is a factor that would affect the numbers, and my analysis doesn’t capture it.
Graham notes that repository costs, monitoring costs and subscriptions would reduce in the event that all of the UK’s research output moves to gold OA. I think this certainly true for repository costs, but these administrative costs would be replaced by payment processing costs within universities for dealing with APC funds etc., which as this report has shown, are much higher than the costs of posting in repositories. Monitoring would still be necessary for the REF, and would be harder as compliance would depend on third parties (publishers), not local services (repositories). There would still be the costs of dealing with the REF processes and policies, too, and these might rise or fall with each new variant of policy across the wealth of funders that authors and institutions have to deal with. So I decided to exclude administrative costs; but if I were to include them, I think it would push the cost up, not down.
Would subscriptions fall if 100K+ articles became available OA? No, not unaided. We’re talking about 6% of research output globally. Why would an institution cancel a journal big deal if they could only access 6% of the articles for free? To say prices will fall ignores the long-standing issue that subscription prices are continuing to rise – a still current trend, and one that’s predicted to continue into 2015. Subscriptions only fall when they are under threat. They are not under threat from local moves to gold OA, not without the kind of interventions being staged by University of Konstanz or by universities in The Netherlands. They might fall in tandem with APC rises if publishers realise en masse that double dipping is a real issue and resolve to follow IoP and RSC, though…
Infrequent investigations 