"But is it peer reviewed?" - Fraud and Science
By Arthur Dobrin | PsychologyToday.com
Cheating in the sciences may be the most serious form of fraud .
Cheating amongst students grabs the headlines one day. Everyone knows about cheating in the business world. Think Libor.
A smaller story about cheating on the inside page of the NY Times is equally upsetting, if only because it comes from an unexpected quarter.
Now the culprits are research scientists.
More than 2,000 papers were analyzed. These were scientific findings that were published in journals but later retracted. When the news first reported the retractions in publications in the biomedical and life sciences fields, the reason offered for the recalls was that the researchers had made honest mistakes.
When looked at more closely, it turns out that three-quarters of the retracted papers were fraudulent. Researchers engaged in active falsification of data, not computational mistakes.
Benjamin Druss, an Emory University professor of health policy, notes that while the number of published research papers is high, the overall percentage is low, about one in 10,000.
Cheating in schools is a serious matter because it undermines the integrity of education. A real education means a student has learned the subject matter. There is no way of knowing what a cheater has learned. More seriously, it makes every potential employer question what an applicant knows and will do.
Cheating in business undercuts the trust that is necessary for efficient economic functions. There is no way to know the true market value of goods or services when the price has been manipulated.
Cheating in the sciences may be the most serious form of fraud. The modern world rests upon society accepting the outcomes of scientific research. Without trust in the scientific procedure, there is no way to know the quacks from the experts. Society can no longer sift through competing claims. Who knows which is snake oil and which a true medicine?
Underlying the streak of cheating in education, business and science is a single factor: the pressure to succeed and the steep cost of failure. One of the authors of the study of fraud in the biomedical and life sciences research, Arturo Casadevail of Albert Einstein College of Medicine, says that the increase in fraudulent papers is due to winner-take-all atmosphere in the field. He notes that getting a paper published in a major journal is the difference between heading a lab and heading for the door.
Read the full article at: psychologytoday.com
Peer review: a flawed process at the heart of science and journals
By Richard Smith | Journal of the Royal Society of Medicine
Peer review is at the heart of the processes of not just medical journals but of all of science. It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away. Famously, it is compared with democracy: a system full of problems but the least worst we have.
When something is peer reviewed it is in some sense blessed. Even journalists recognize this. When the BMJ published a highly controversial paper that argued that a new `disease, female sexual dysfunction, was in some ways being created by pharmaceutical companies, a friend who is a journalist was very excitednot least because reporting it gave him a chance to get sex onto the front page of a highly respectable but somewhat priggish newspaper (the Financial Times). `But, the news editor wanted to know, `was this paper peer reviewed?. The implication was that if it had been it was good enough for the front page and if it had not been it was not. Well, had it been? I had read it much more carefully than I read many papers and had asked the author, who happened to be a journalist, to revise the paper and produce more evidence. But this was not peer review, even though I was a peer of the author and had reviewed the paper. Or was it? (I told my friend that it had not been peer reviewed, but it was too late to pull the story from the front page.)
WHAT IS PEER REVIEW?
My point is that peer review is impossible to define in operational terms (an operational definition is one whereby if 50 of us looked at the same process we could all agree most of the time whether or not it was peer review). Peer review is thus like poetry, love, or justice. But it is something to do with a grant application or a paper being scrutinized by a third partywho is neither the author nor the person making a judgement on whether a grant should be given or a paper published. But who is a peer? Somebody doing exactly the same kind of research (in which case he or she is probably a direct competitor)? Somebody in the same discipline? Somebody who is an expert on methodology? And what is review? Somebody saying `The paper looks all right to me, which is sadly what peer review sometimes seems to be. Or somebody pouring all over the paper, asking for raw data, repeating analyses, checking all the references, and making detailed suggestions for improvement? Such a review is vanishingly rare.
What is clear is that the forms of peer review are protean. Probably the systems of every journal and every grant giving body are different in at least some detail; and some systems are very different. There may even be some journals using the following classic system. The editor looks at the title of the paper and sends it to two friends whom the editor thinks know something about the subject. If both advise publication the editor sends it to the printers. If both advise against publication the editor rejects the paper. If the reviewers disagree the editor sends it to a third reviewer and does whatever he or she advises. This pastichewhich is not far from systems I have seen usedis little better than tossing a coin, because the level of agreement between reviewers on whether a paper should be published is little better than youd expect by chance.1
That is why Robbie Fox, the great 20th century editor of the Lancet, who was no admirer of peer review, wondered whether anybody would notice if he were to swap the piles marked `publish and `reject. He also joked that the Lancet had a system of throwing a pile of papers down the stairs and publishing those that reached the bottom. When I was editor of the BMJ I was challenged by two of the cleverest researchers in Britain to publish an issue of the journal comprised only of papers that had failed peer review and see if anybody noticed.
I wrote back `How do you know I havent already done it?
THE DEFECTS OF PEER REVIEW
So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.
Slow and expensive
Many journals, even in the age of the internet, take more than a year to review and publish a paper. It is hard to get good data on the cost of peer review, particularly because reviewers are often not paid (the same, come to that, is true of many editors). Yet there is a substantial `opportunity cost, as economists call it, in that the time spent reviewing could be spent doing something more productivelike original research. I estimate that the average cost of peer review per paper for the BMJ (remembering that the journal rejected 60% without external review) was of the order ofŁ 100, whereas the cost of a paper that made it right though the system was closer to Ł1000.
The cost of peer review has become important because of the open access movement, which hopes to make research freely available to everybody. With the current publishing model peer review is usually `free to authors, and publishers make their money by charging institutions to access the material. One open access model is that authors will pay for peer review and the cost of posting their article on a website. So those offering or proposing this system have had to come up with a figurewhich is currently between $500-$2500 per article. Those promoting the open access system calculate that at the moment the academic community pays about $5000 for access to a peer reviewed paper. (The $5000 is obviously paying for much more than peer review: it includes other editorial costs, distribution costsexpensive with paperand a big chunk of profit for the publisher.) So there may be substantial financial gains to be had by academics if the model for publishing science changes.
There is an obvious irony in people charging for a process that is not proved to be effective, but that is how much the scientific community values its faith in peer review.
People have a great many fantasies about peer review, and one of the most powerful is that it is a highly objective, reliable, and consistent process. I regularly received letters from authors who were upset that the BMJ rejected their paper and then published what they thought to be a much inferior paper on the same subject. Always they saw something underhand. They found it hard to accept that peer review is a subjective and, therefore, inconsistent process. But it is probably unreasonable to expect it to be objective and consistent. If I ask people to rank painters like Titian, Tintoretto, Bellini, Carpaccio, and Veronese, I would never expect them to come up with the same order. A scientific study submitted to a medical journal may not be as complex a work as a Tintoretto altarpiece, but it is complex. Inevitably people will take different views on its strengths, weaknesses, and importance.
So, the evidence is that if reviewers are asked to give an opinion on whether or not a paper should be published they agree only slightly more than they would be expected to agree by chance. (I am conscious that this evidence conflicts with the study of Stephen Lock showing that he alone and the whole BMJ peer review process tended to reach the same decision on which papers should be published. The explanation may be that being the editor who had designed the BMJ process and appointed the editors and reviewers it was not surprising that they were fashioned in his image and made similar decisions.)
Sometimes the inconsistency can be laughable. Here is an example of two reviewers commenting on the same papers.
"Reviewer A: `I found this paper an extremely muddled paper with a large number of deficits"
"Reviewer B: `It is written in a clear style and would be understood by any reader."
Read the full article at: rsmjournals.com
Trust me - Im a Doctor
The drugs dont work: a modern medical scandal
$28 billion health fund backed by Bill Gates and Bono is investigated for fraud
Piers Corbyn: Delusion & Fraud: Global warming nonsense lobby to collapse (Video)
The Mysterious Laptop Documents: "Evidence" of Iran Nuclear Weapons Program May Be Fraudulent
US physics professor: "Global warming is the greatest and most successful pseudoscientific fraud I have seen in my long life"
Pfizer to pay $142M for drug fraud
The Fraud of Cap & Trade
Latest News from our Front Page
Amid Russia tensions, US nuclear bombers to conduct military drills in Sweden
The Pentagon is planning to send nuclear bombers to Sweden for a military exercise next month amid growing tensions with Russia over the Ukraine crisis.
The warplanes, the B-52 Stratofortress, will participate in a naval exercise on June 13, Swedish general Karl Engelbrektson said.
They are set to fly from the United States nonstop and simulate a drop of anti-ship mines near ...
'Netanyahu to US: Give 50% more money, we'll shut up'
Israeli Prime Minister Benjamin Netanyahu is asking the United States to provide Tel Aviv 50 percent more money for weapons and âweâll shut upâ on Iran nuclear talks, an author and investigative journalist in Philadelphia says.
Dave Lindorff made the remarks in a phone interview with Press TV on Wednesday while commenting on a report which says Israel has asked Washington ...
Britain To Outlaw "Hate" and "Extremism"
UK home secretary Theresa May : "But what we're talking about is they key values that underline our society and are being undermined by the extremists. Values like democracy, a belief in democracy, a belief in the rule of law. A belief in tolerance ...eh... for other people. Equality and acceptance for other people's faith and religions.
One of the great ...
Killer robots will leave humans 'utterly defenceless' warns professor
Robots, called LAWS â lethal autonomous weapons systems â will be able to kill without human intervention.
Killer robots which are being developed by the US military âwill leave humans utterly defencelessâ, an academic has warned.
Two programmes commissioned by the US Defense Advanced Research Projects Agency (DARPA) are seeking to create drones which can track and kill targets even when ...
Here's how much corporations paid US senators to fast-track the TPP bill
Critics of the controversial Trans-Pacific Partnership are unlikely to be silenced by an analysis of the flood of money it took to push the pact over its latest hurdle.
A decade in the making, the controversial Trans-Pacific Partnership (TPP) is reaching its climax and as Congress hotly debates the biggest trade deal in a generation, its backers have turned on the ...
|More News » |