Peer Review: Still The Gold Standard?

by Research Insights0 comments

Share this page:

When the practice began 350 years ago, Peer Review was designed to validate academic work and improve the quality of published data. You may still think it does.

We cite a famous case of scientific fraud to show how ineffective peer review can be, and how difficult it is to reverse published results, even when fraud is proven and lives are at risk. While it began way back in 1999, this case still has impact today.

‘The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding … 

Richard Horton, Editor, The Lancet

 Spectacular Results

Erasmus Medical Centre, Rotterdam

In 1999, Dr Don Poldermans, a cardiology researcher at the Erasmus Medical Centre (EMC) in Rotterdam, published data from six clinical trials in the respected, peer-reviewed New England Journal of Medicine (NEMJ).

These results showed that, in 91% of cases, perio-operative beta-blocker therapy protected the hearts of patients undergoing high-risk, non-cardiac surgery. Based on these impressive results, this therapy became a European Society of Cardiology (ESC) ‘best practice’ guideline in 2009.

Best Practice Gone Bad

By 2011, British cardiologists Drs Graham Cole and Darrel Francis found no other studies produced similar results, so they re-analysed the original trial data. Not only did they find that ‘in a large number of cases the trial outcomes were fictional’, the original data showed that beta-blockers actually increased the risk of death during operations by 27%.

They alerted the ESC by letter, which initially posted their article but then took it down, citing the fact that it hadn’t been peer-reviewed. This was unexpected, as  only major studies and trials are subjected to peer review. As Coles and Francis later stated: ‘There seems to be no clause permitting ESC experts to raise the alarm when needed, even to prevent loss of life’.

Poldermans was dismissed from the EMC in 2011 but the ESC did not downgrade its best practice guide until 2014. By that time, Cole and Francis estimated, some 800,000 European had lost their lives as a result of the therapy based on Poldermans’ fraudulent data.

‘There seems to be no clause permitting ESC experts to raise the alarm when needed, even to prevent loss of life’.

Drs Cole and Francis, Letter to Erasmus Medical Centre

Peer Review or Peer Protection?

‘How could this happen when this research was peer-reviewed?’ you may ask. We, the public, imagine that the peers are like auditors going over the data with a fine-toothed comb. That is not so.

‘…the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.’  Richard Horton, Editor, The Lancet

According to Richard Smith, Former Editor of the British Medical Journal, it’s more like ‘this paper looks OK to me’. Very rarely, if ever, do peers re-analyse the original data as Cole and Francis did.

More pertinent perhaps, to be chosen to review, peers must work in similar areas of research as the authors. This can be problematic when peers often co-author papers or sit on the same committees with the authors. This is what happened in the Poldermans’ case.

Cardiobrief editor Larry Huston pointed out that one of the Poldermans reviewers, Jeroen Bax, co-authored hundreds of papers with Don Poldermans, and that, together, they co-authored the beta blocker guideline. Poldermans was also the chair of the writing committee, and Bax the chair of the ESC Cardiology Practice Guidelines Committee.

Huston adds that Cole and Francis focused on the broader implications of the scandal. ‘They write about a culture of neglect in which few if any participants have anything to gain by finding or reporting scientific misconduct. They cite numerous examples in which misconduct has been alleged but the responsible actors– authors, home institutions, journals, and medical societies– have responded in only the most minimal and nonaggressive fashion. The portrait they paint is of a scientific and medical establishment devoted to not rocking the boat.’

‘They (Cole & Francis) write about a culture of neglect in which few if any participants have anything to gain by finding or reporting scientific misconduct.’

Larry Huston, Editor, Cardiobrief

Misconduct or Dangerous Fraud?

The Erasmus Medical Centre conducted 2 enquiries into Poldermans’ research starting in 2011. It found many instances of scientific misconduct and dismissed Poldermans from the EMC in 2011, yet the final report may surprise you. It states that ‘Poldermans and Erasmus MC are in full agreement that the misconduct did not lead to health damage to patients.’ As a footnote, in 2021, the NEJM has still not retracted the original Poldermans’ articles.

‘Poldermans and Erasmus MC are in full agreement that the misconduct did not lead to health damage to patients.’

Final Report, Erasmus Medical Centre
9 Secrets

9 SECRETS TO LIVING IT UP FOR LONGER

For Aussie boomers only

References

What is Peer Review; Elsevier
Perioperative β blockade: guidelines do not reflect the problems with the evidence
Medicine Or Mass Murder? Larry Huston, Cardiobrief
A ‘Disappeared’ Article, Finally Published, Finds A Desultory Response To Scientific Misconduct, Larry Houston, Cardiobrief
Peer review: a flawed process at the heart of science and journals; Richard Smith; Journal of the Royal Society of Medicine.

Kim Brebach

Kim Brebach

Hi, I’m Kim Brebach, boomer, information researcher, technical writer and Joiner of Dots at M&M. In my spare time, I review wines and love to cook.

[]
×
Like an EXTRA 5% OFF?

Claim your discount now

No thanks. I'm happy to pay the offered price. *You can unsubscribe at any time.