Peer Review - Politics of Science?

Readers' Comments

I would like to share a recent email received from

ALLIANCE FOR HUMAN RESEARCH PROTECTION (AHRP)
Promoting Openness, Full Disclosure, and Accountability
www.ahrp.org

An original essay by Dr. Marc Girard, a mathematician and physician who serves on the editorial board of Medicine Veritas, (The Journal of Medical Truth) a peer reviewed, open source journal "Dedicated to Leveling out the Medical Playing Field."

Dr. Girard is not impressed with the disingenuous displays of "outrage" at individual scientists who got caught committing fraud-e.g., Hwang Woo Suk:

"the main advantage of individual failures," he writes, "is that they give a pretext for virtuous protestations from those whose success is based upon systemic failures." Scientific integrity: "Truth" versus method

Vera Hassner Sharav

~~~~~~~~~

Dr. Marc Girard

Although I am not myself a geneticist, the German journal, GID, requested me to comment about the Hwang story (2), probably because I have recently written that due to the dishonest compromises of most experts "medical research has exerted a disastrous influence over other branches of science" (1). Fortunately for me, and alas for Science, a recent story of gross fraud concerning anti-inflammatory agents (3) demonstrates that the problem goes far beyond genetics (or South Korea.). And it is also intimately related to the specificity of science as a human activity.

Whatever the philosopher K. Popper may have said, the pursuit of Truth is by no means characteristic of the sciences: an honest judge who sentences (possibly, to death) an accused, a sincere lover who wants to be sure that his/her feelings are reciprocal, are far more obsessed with "truth" than a researcher in meteorology who knows that the rigour of his observations and the sophistication of his mathematical models notwithstanding, his results are marked by a strong degree of uncertainty. And who would deny that a meteorologist behaves as a scientist far more than a judge or a lover?

The specificity of science and, more precisely, its greatness, is not a quest for Truth, but far more simply: the quest for the perfect method. What sets scientists apart is the fascination of shedding our innate subjectivity in order to provide others with the means of replicating our observations and results. In this quest for sharing with others - which also takes the form of a request for their critical feedback - the system of peer-review, in one form or another, has always been pivotal for the credibility of science: every one of us knows its limitations and is able to quote a number of its historical failures, but the sheer reality is that nobody has ever been able to conceive a more reliable alternative.

It is probably not true that the history of sciences includes a number of "unrecognised" geniuses: easily accepted or not, those researchers who made a significant contribution to a science were those who eventually managed to introduce their ideas and make them recognized by their peers. This is neither an easy nor a democratic process, but this has been the normal way of being "a scientist" as long as the ultimate target has been to share with others: after all, being regularly rejected by peers in a kind of methodological failure. In its essence, scientific activity has always been asceticism, and certainly not a means for personal recognition or social promotion. And in Western history, if the Christian religion has most often been an obstacle to sciences, this was not because scientific "truth" was an alternative to God (which it is not): this was because scientific asceticism - as an appeal to go out of one's ego and to reach a certain order of objectivation, if not of objectivity - was a genuine alternative to that rooted in the notion of original sin. Another form of virtue.

In the meantime, however, scientific research has become a road for individual success - which was probably not the main impulse of our great ancestors such as Galileo, Pasteur, Darwin, Freud or Planck. And the pressure to gain success has been such that it has circumvented our control system. If co-authors, within the functioning of one given scientific team, are demonstrably unable to guarantee the reliability of their own results - as is obviously the case in recent stories (2;3) - who will seriously believe that external peers, who devote a maximum of a few hours to the review of a manuscript, will do better?

Experience, on the contrary, suggests that the situation is even worse. It is a pity to compare the volume of editorials devoted to piously denounce individual failures versus the scant attention editors spend for detecting those fallacies which have a genuine impact on society. After all, the presumed falsifications of Hwang did not trigger any change in the practical management of people, which is not the case with a number of appalling investigations "turning us all into patients" via the fallacies of high blood pressure, menopause disorders, high cholesterol, etc. (4) - for which a dime a dozen papers have been published in the greatest journals, which is precisely the background of their success.

If a "scientific" journal is not able to recognize as fraudulent, an investigation (by one European researcher) whose 250 patients were supposedly born on the same day, who will be able to rely on huge epidemiological investigations-including thousands of patients monitored during years in the farthest parts of developing countries?

Whereas counterfeit drugs from East or far-East are becoming an obsession of pharmaceutical leaders, how should we interpret the concomitant tendency of the same firms to export their most crucial clinical trials to the same geographical areas? And how do we explain the depressingly lax reviewing process by leading medical journals that publish such studies nonetheless? Is it not far more simple to control the chemical content of a pill than to check millions of data recorded over years in thousands of unidentified patients? As I said some time ago in a meeting organised by the pharmaceutical industry: the main advantage of individual failures is that they give a pretext for virtuous protestations from those whose success is based upon systemic failures.

This is a critical moment: if the system of peer-review is not any longer able to guarantee the reliability of scientific research, this means that science has lost its way. The reason for this disaster is too clear: the power of money. In academic institutions, the current dynamics of research is more favourable to the ability of getting grants - collecting money and spending it - than to scientific imagination or creativity. The business of publishing is fuelled by a continual production of new data of problematic interest, whereas there is no place now to interpret, correct or synthesize previous results . A striking illustration of this state of affairs is the absurd rule of some medical journals to limit the time for submitting correspondence on a paper to 15 days. If science is the target, there is no deadline to put an old result in a new perspective - or more simply to detect an inconsistency in a previous investigation.

Thus, besides precautionary principles such as the declaration of interests, why do we not think about a simple measure the reach of which could be considerable: a systematic rejection of a work by journals when its budget would have gone beyond a certain limit? Like the systematic rejection of biomedical research which did not comply with the requirement of participants' informed consent, such a measure would mean that sciences have greater values than a presumed "truth" whatever its cost: i.e. the conviction that too much money is the surest way to ruin the fundamental prerequisite of peer-control.

References

1. Girard M. Reformulating the principles of Hippocrates. Medical Veritas 2005;2:682.

2. Anon. Writing a new ending for a story of scientific fraud. Lancet 2006;367:1.

3. Horton R. Expression of concern: non-steroidal anti-inflammatory drugs and the risk of oral cancer. Lancet 2006 Jan 21;367(9506):196.

4. Moynihan, R; Cassels, A. Selling Sickness. How drug companies are turning us all into patients. Crows Nest: Allen & Unwin; 2005.

5. Feyerabend, P. Against Method. Humanities Press; 1975.

6. Sadovnick AD, Scheifele DW. School-based hepatitis B vaccination programme and adolescent multiple sclerosis [letter]. Lancet 2000 Feb 12;355(9203):549-50.

7. Wraith DC, Goldman M, Lambert PH. Vaccination and autoimmune disease: what is the evidence? Lancet 2003 Nov 15;362(9396):1659-66.

Posted by: Sepp on January 31, 2006 04:18 PM

 

Here is an interesting article that appeared in USA Today - January 2007:

Is this the end of the scholarly journal?

By Gregory M. Lamb, The Christian Science Monitor

Scientific advances sometimes come as lightning flashes of inspiration. But when scientists sit down to record and take credit for what they've found, they still use much the same method they have for decades – an article published in a scholarly journal.
But science's hidebound traditions are changing. The Internet has opened up new forms of publishing in which anyone in the world can find and read a scientific paper. And papers themselves are becoming more interactive, leading readers to the underlying data, videos, and discussions that augment their value. With blogs and e-books providing easy means of self-publishing, some observers are speculating that scholarly journals and their controversial system of peer reviews may not be needed at all.

"The traditional journal publishing medium we've grown used to really needs to evolve and change because that's not the way people are accessing information," says Mark Gerstein, a professor of biomedical informatics at Yale University in New Haven, Conn. Dr. Gerstein cowrote an article, "The Death of the Scientific Paper," which appeared last year on The-Scientist.com, an online science magazine.

If the hopes of innovators bear fruit, scientific advances will come ever more quickly as online publishing makes past research easier to access and share widely.

Two new scientific publications, both available only online, may signal what's ahead. The PLoS ONE (plosone.org), a journal begun by the Public Library of Science (PLoS) last month, aims to put as many new scientific articles as possible on the Internet to be read by anyone, free of charge. The Journal of Visualized Experiments, or JoVE (www.myjove.com), is a kind of YouTube for researchers. It operates on the theory that a short video showing how an experiment is done is better than thousands of words that attempt to describe it.

At PLoS ONE, which aspires to be a general science journal along the lines of Science and Nature, the papers themselves are only a starting point. Readers can annotate, comment on, and critique the findings: Their contributions become permanently attached to the original article. At least one commentator has likened this process to a kind of "electronic Talmud," in which the original document receives elaborate commentary and discussion that over time adds greatly to its value.

In coming months, says Chris Surridge, the managing editor of PLoS ONE, readers also will be able to rate papers on their quality, such as how surprising or groundbreaking the results were – much in the way Netflix subscribers rate movies they rent using one- to five-star ratings. In this sense, PLoS ONE is moving toward a Web 2.0 model, which focuses on user-generated content strategies already used by websites such as Digg.com, Slashdot.org, or Amazon.com.

For years, traditional "peer review" has come under fire. A jury of three experts, the peer reviewers, assess each article and recommend only those that they feel represent the most significant new work. At many elite scientific journals, fewer than 10 of the articles submitted are accepted. Many of the rejected articles eventually travel down the "food chain" to be published in a plethora of less prestigious (and less noticed) specialty journals.

A year ago, the respected U.S. journal Science was forced to retract two papers it had published about stem cells. The articles had been submitted by a South Korean team led by Hwang Woo-Suk. Peer reviewers, as well as the editors, had failed to detect the fraud.

In general, peer reviewers, themselves researchers pressed for time, don't try to re-create experiments and rarely ask to see the raw data that supports a paper's conclusions. While peer review is expected to separate the wheat from the chaff, it's "slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless for detecting fraud," summed up one critic in BMJ, the British medical journal, in 1997.

"There's a lot of discussion [in the scientific community] about how peer review doesn't work," Mr. Surridge says. "It's not a great way to decide [what to publish]. It's just the only way we have at the moment."

PLoS ONE takes a different tack. While articles receive a basic screening, they don't have to attain the standard of representing groundbreaking work in order to be published. An article only has to be based on solid science. The idea is that the more valid research is published, the better, as it contributes to an online database.

"If it is science, [if] it is well done, [and if] it provides a valuable contribution to scientific literature, we can publish it," Surridge says. He expects about two-thirds of those papers submitted to PLoS ONE to be accepted.

Since its launch Dec. 20, PLoS ONE has published well over 100 papers and expects to publish 15 to 20 more per week. Readers access the articles for free. PLoS ONE pays its way by charging authors $1,250 to publish an article. While that might seem a barrier to publication, Surridge says most research is financed by grants or large institutions, meaning individual scientists rarely have to pay themselves. But just in case, PLoS ONE is waiving the fee for any authors who request it.

Moshe Prisker and Nikita Bernstein are taking another approach, using the Web's ability to deliver video easily with JoVE. While scientific concepts can be very simple, "the actual doing of the [laboratory] experiments is very difficult," says Dr. Prisker, a neural-stem-cell researcher at Massachusetts General Hospital in Boston.

Traditionally, he says, scientists have resorted to wandering from lab to lab asking, "Do you know how to do this or do that? They're basically asking someone with previous experience to show them."

In its first three months, JoVE has posted 18 videos ranging from five to 15 minutes showing techniques such as "Nuclear transfer in mouse oocytes." They get a modest vetting from scientists in the field to make sure they are sound. The site is free to view and charges nothing to post videos. Prisker says that he and Mr. Bernstein, both volunteers, hope that the journal will someday pay for itself through ads from manufacturers of lab equipment (the "Google model," he says).

Other journals are beginning to employ video in some articles, but JoVE is the first to make video images the primary means of conveying information. Brief articles, voice-overs, or captions accompany the moving images.

"Video gives you this ability for unambiguous representation of experiments," Prisker says. "I'm sure that video publication will become a significant force [in online journals]." Nearly all of the early reaction to the fledgling journal has been positive. Visitors to the JoVE website say, "This should have been done long ago," Prisker says.

Since the early days of the Web, observers have speculated that scientists might simply post new research on their own or in communal websites and let search engines find it, thereby bypassing the peer-reviewed journals altogether. If the research proves valuable, other sites will link to it, and the results would be "published" far faster than waiting for a journal to accept them.

Already, an online database called arXiv (www.arXiv.org), hosted by Cornell University in Ithaca, N.Y., contains more than 400,000 scientific papers posted by their authors without peer review. (The papers often appear later in peer-reviewed journals). Its comprehensiveness makes arXiv (pronounced "archive") a valuable tool, Gerstein, the Yale researcher, says. If someone claims to make a new discovery, anyone can search this database and say, "No, you didn't. It's in the arXiv."

Nonetheless, Gerstein says he thinks scientific journals, and some kind of peer review, will be around for a long time. Publishing in prestigious journals is "deeply intertwined with [scientists'] reputations and their promotions," he says. "You still want to get the stamp of approval of a journal."

Posted by: Sepp on January 24, 2007 12:21 PM

 

Name: Email Address: URL: Comments: Remember info? Security code: Please enter the security code displayed on the above grid Due to our anti-spamming policy the comments you are posting will show up online within few hours from the posting time.

	The Individual Is Supreme And Finds Its Way Through Intuition