A Case for Open Peer Review for Clinical Trials

A few weeks ago I played the part of an expert witness in the Medical Journalist Association’s mock trial, Trials on Trial. The charge was: “Is the current system of publishing clinical trials fit for purpose?” The jury’s verdict was a resounding ‘no’. You can read more about the event in Jane Feinmann’s write up on the BMJ Group Blogs.

As an expert witness on a panel tasked with giving evidence on different elements of publishing clinical trials, I was asked “how does open peer review work and does it address all the flaws of the current system of publishing clinical trials?”

I gave my evidence on why I believe that open peer review is the best system of peer review for publishing clinical trials. I explain my arguments below.

It’s vital that that articles reporting results of clinical trials have been appropriately peer reviewed as they form the basis of doctors’ clinical decision making. Traditionally, this process has taken place anonymously (‘closed peer review’) so only the editor knows the reviewers’ identities.

What is open peer review?

Some journals – including the BioMedCcentral-series medical journals, Trials, Journal of Negative Results in BioMedicine and BMJ Open – operate a system of open peer review.

It’s a peer review model that involves two levels of openness. The first level is that authors know who reviewed their article, and the second is that readers know too – if the article is published, a pre-publication history is posted online alongside the article. This contains all versions of the manuscript, the reviewers’ signed reports and the authors’ responses to reviewers’ and editors’ comments.

If you want to know more about the history of open peer review, this blog from F1000 Research gives a nice description and timeline.

What are the problems with anonymous peer review of clinical trials?

There are some drawbacks with closed peer review for clinical trials. Peer review is akin to a ‘black box’ – no-one outside of the journal knows how decisions were made. There’s less accountability for the decision and also room for poor practice and misconduct – editors, for example, may choose the wrong peer reviewers, or reviewers may review manuscripts on which they have an undeclared conflict of interest and potentially stall, or even prevent, publication.

How does open peer review deal with these issues?

It may not be a panacea, but open peer review goes some way to solving these problems. Most importantly it increases transparency – something that is key to the whole process of publishing clinical trials.

Increased transparency increases accountability:

• Peer reviewers are more accountable for their reports as they’re attaching their name to them.
• Editors are also more accountable – their decisions are no longer made behind closed doors. Anyone can see whether peer review was appropriate.

It also brings transparency to the thorny issue of competing interests – rather than simply relying on the editor’s opinion of whether something counts as a competing interest, reviewers’ statements of competing interests in the pre-publication history allows the reader to make their own judgments.

Transparency in open peer review allows doctors to make more informed decisions – they can look at whether research was reviewed appropriately and therefore to what extent they believe the results.

Does it work?

There’s little empirical evidence on open peer review, but from what little there is available, we do know that telling reviewers that their names will be revealed to authors (van Rooyen et al. 1999) and that their reports may be posted online (van Rooyen et al. 2010) does not decrease the quality of their reports.

In our recent research, we compared the quality of peer reviewers’ reports on an open peer review journal (BMC Infectious Diseases) and a closed peer review journal (BMC Microbiology), and found an increase in the quality of reviewers reports on open peer review. This improved quality was  specifically seen in the areas of comments on the methods and study design, supplying evidence to support comments, and constructiveness.

We also know that open peer review is feasible. As argued by Fiona Godlee, experience from her time at both BioMed Central and the BMJ shows us that the model works.

Summing up

Open peer review increases transparency by bringing the peer review process into the public domain to help doctors make informed decisions. It not only provides information on how a decision was made on a specific article, but it also provides data for much needed research into peer review allowing us to better understand the process by which we evaluate clinical trials on which we base clinical decisions.

Despite the jury’s verdict that the current system of publishing clinical trials is not fit for purpose, I would argue that open peer review does go some way to solving some of the issues.