Peer Review - Politics of Science?
According to a recent article on BushGreenwatch.org, the White House is looking for ways to more closely control what scientists are allowed to say in studies that are to be used by the US government in forming policy in the areas of health and the environment.
The peer review system, whereby a scientific article is scrutinized by a scientist's colleagues - actually often by an anonymous selection of "guardians of orthodoxy" who work with the major scientific magazines - has already come under fire for hampering the open scientific discussion of new ideas. With government getting into the act to "select" the peer reviewers, we may see science degrading into a political tool, rather than a way to ascertain the truth about issues of interest.
Not that government is new to attempts at "damage control" by changing scientific findings. Here a recent report from AmericanProgress.org.
"DOCTORING DATA: A new report reveals that political appointees in the Department of Health and Human Services (HHS) doctored a study that revealed racial disparities in healthcare. The report, commissioned by Rep. Henry Waxman (D-CA) and seven colleagues, compares the draft report prepared by HHS scientists to the final version, edited by political appointees. In an effort to minimize the problem, political appointees deleted key sections of the report. Deleted: conclusion by HHS scientists that healthcare disparities are "national problems." Deleted: findings by HHS scientists on the social costs of healthcare disparities (including lost productivity, needless disability and early death). Deleted: key examples of health care disparities, including findings that racial and ethnic minorities are more likely to be diagnosed with late-stage cancer, die of HIV and be subjected to physical restraints in nursing homes."
January 22, 2004
White House Seeking Control Over Scientific Peer Review of Environmental, Health Research Which Shapes Federal Policies
Environmental and health studies conducted for or used by the federal government would require White House approval before their release, under a proposal now under review at the Office of Management and Budget (OMB). The plan would also give the White House authority to select which scientists take part in the system known as peer review -- the process by which fellow researchers evaluate the validity and reliability of studies before they are published.
Critics fear such a plan would undermine the impartiality of research that guides government policies and regulations. For example, it would open the door for the Administration to hand-select industry-friendly scientists to review studies that investigate the safety of chemicals in our food and consumer products, or studies that examine the environmental impact of energy plant emissions. The White House has frequently expressed its commitment to easing regulations for American industries. 
In a January 9 letter to the OMB, 20 former top federal agency officials, from both Democratic and Republican administrations, urged the White House to drop its proposal. The letter -- signed by former EPA Administrators Carol Browner and Russell Train; former Secretary of Labor Robert Reich; former Assistant Secretaries for Occupational Safety and Health Eula Bingham and Gerard Scannell; and others -- warned that the proposal, "in its current form, could damage the federal system for protecting public health and the environment." 
Currently, each federal agency controls peer review of its own projects. The government's rules to ensure research quality are already less stringent than those used by leading biomedical journals. For example, these journals require authors to disclose who paid for the research; and the journals will only publish studies done under contracts in which the investigators have the right to publish regardless of the results. Federal agencies do not have these requirements, nor do they consistently attempt to find out who paid for the studies. 
Far from ensuring the validity of the peer review process, the plan's critics assert that allowing the White House to control it would only add a layer of politics to what should be a purely scientific process. 
 "White House Seeks Control on Health, Safety," St. Louis Post-Dispatch, Jan. 11, 2004.
 Letter to Joshua B. Bolton, Director of the OMB from 20 former agency officials, Jan 9, 2004.
 Michaels D, Wagner W. Disclosure in Regulatory Science. Science. 2003; 302:2073.
 "Peer Review Plan Draws Criticism," Washington Post, Jan. 15, 2004.
- - -
How to Fix Peer Review
Separating its two functions - improving manuscripts and judging their scientific merit - would help
(published in The Scientist)
By David Kaplan
Despite its importance as the ultimate gatekeeper of scientific publication and funding, peer review is known to engender bias, incompetence, excessive expense, ineffectiveness, and corruption. A surfeit of publications has documented the deficiencies of this system.1-4 In September, the fifth in a series of international congresses concerned with how peer review can be improved will convene in Chicago. Yet so far, in spite of the teeth gnashing, nothing is being chewed.
Investigation of the peer-review system has failed to provide validation for its use.1 In one study, previously published articles were altered to disguise their origin and resubmitted to the journals that had originally published the manuscripts.5 Most of these altered papers were not recognized and were rejected on supposed "scientific grounds." Other investigators found that agreement among reviewers about whether specific manuscripts should be published was no greater than would be expected by chance alone.6
Peer review subsumes two functions. First, peer reviewers attempt to improve manuscripts by offering constructive criticisms about concrete elements such as the application of a technique, the strength of results, or the cogency of an argument. The second function of peer review is to render a decision about the biological significance of the findings so that the manuscript can be prioritized for publication. I propose reforming peer review so that the two functions are independent.
Review of a manuscript would be solicited from colleagues by the authors. The first task of these reviewers would be to identify revisions that could be made to improve the manuscript. Second, the reviewers would be responsible for writing an evaluation of the revised work. This assessment would be mostly concerned with the significance of the findings, and the reviewers would sign it.
After receiving the final assessments from several different reviewers, the authors could decide to submit to a journal, sending the manuscript and the signed reviews together. The editors, carrying out the second function of peer review, would then decide to publish or not based solely on this material. The reviewers' identities would be revealed in the publication.
I believe there would be several significant effects of this change in peer review. First, the authors would submit only positive assessments. Consequently, reviews would emphasize why a manuscript should be published instead of why it shouldn't be. Second, investigators would be less likely to publish insignificant findings. They would have to ask colleagues to put their names on the manuscript; consequently, the tendency would be to ask for support for more complete and more compelling sets of findings.
Third, reviewers would be forced to account for their comments. They could not perform just a cursory look without the authors realizing the review was not insightful and did not represent an honest effort. Fourth, although it would be possible to have close friends and relatives review a manuscript, the editors would see who was supporting publication. In their deliberations, the editors would consider the breadth of the reviewers and their relationships to the authors and to the conceptualization promulgated in the manuscript.
Fifth, the editors would be free from adjudicating between authors and reviewers. They could concentrate on the specific arguments put forth for publication. Moreover, the process would be considerably streamlined, since there would be no need to send the manuscript out for review. This revision of peer review would change the incentives for all involved. The authors would tend to publish results that represent more complete findings and be more satisfied with the outcome, because they could exert lots of control over the review process. The reviewers would tend to be more honest in their evaluations, not wanting to praise work they consider flawed, because their names would be attached to it. Reviewers would not give a cursory and willfully negative evaluation, because the authors could simply not forward their comments. It would be in the reviewers' best interests to help improve manuscripts that have flaws but are potentially important.
The editors would emphasize publication of manuscripts that have the broadest support among scientists in the relevant community or that have the greatest potential to influence the community. Their jobs would be easier because the number of manuscripts submitted would be fewer, although of more substance. This tendency would be facilitated by editors' publicizing the stringent acceptance requirements. For example, editors could request manuscripts with support from reviewers from the same institution and from other institutions. They could request reviewers in the same field and reviewers in related fields.
Peer review is broken. It needs to be overhauled, not just tinkered with. The incentives should be changed so that: authors are more satisfied and more likely to produce better work, the reviewing is more transparent and honest, and journals do not have to manage an unwieldy and corrupt system that produces disaffection and misses out on innovation.
1. T Jefferson et al, "Measuring the quality of editorial peer review," J Am Med Assoc 2002, 287: 2786-90. [Publisher Full Text]
2. PA Lawrence "The politics of publication," Nature 2003, 422: 259-61. [PubMed Abstract][Publisher Full Text]
3. DF Horrobin "The philosophical basis of peer review and the suppression of innovation," J Am Med Assoc 1990, 263: 1438-41. [Publisher Full Text]
4. M Enserink "Peer review and quality: A dubious connection?" Science 2001, 293: 2187-8. [PubMed Abstract][Publisher Full Text]
5. DP Peters, SJ Ceci "Peer-review practices of psychological journals: The fate of published articles, submitted again." Behav Brain Sci 1982, 5:187-96.
6. PM Rothwell, CN Martyn "Reproducibility of peer review in clinical neuroscience," Brain 2000, 123: 1964-9. [PubMed Abstract][Publisher Full Text]
President Bush's administration distorts scientific findings and seeks to manipulate experts' advice to avoid information that runs counter to its political beliefs, a private organization of scientists asserted yesterday...
The Scientist - FEATURE (February 2006):
Is Peer Review Broken?
Submissions are up, reviewers are overtaxed, and authors are lodging complaint after complaint about the process at top-tier journals. What's wrong with peer review?
Peer Review Is Mediocre at Best
Regarding bias in "science" and the utter balderdash that passes for peer-reviewed science, I sometimes feel like a lone voice in the wilderness. Well, thank God -- another blogger has thrown down the gauntlet on the topic.
"Most people think peer review is some infallible system for evaluating knowledge. It's not. Here's what peer review does not do: it does not try to verify the accuracy of the content. They do not have access to the raw data. They don't re-run the statistical calculations to see if they're correct. They don't look up references to see if they are appropriate/accurate."
Debating Peer Review and its more open Alternatives
"The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong."
A re-review of peer review: Leading journal looks to end the 'review nightmare'
Speaking of the launch of the policy, Miranda Robertson said 'Of course journals must do their best to ensure that the research they publish is valid, but the primary function of a journal editor is to promote the dissemination of research results, not to obstruct it. I hope this experiment will show that referees, authors and journals can work together to accelerate the publication of important research.'
Three myths about peer review
What's the future of scientific peer review? The way science is communicated is currently changing rapidly, leading to speculation that the peer review system itself might change. For example, the wildly successful physics preprint arXiv is only very lightly moderated, which has led many people to wonder if the peer review process might perhaps die out, or otherwise change beyond recognition.
Christopher Turner: Peer Review is open to Fraud
As long as publications and grant support are used as critical benchmarks for career progress, the peer review system will remain vulnerable to fraud. There is growing dissatisfaction with this system and the emergence of online "open access" models suggest something better is on the horizon, though for some it may not arrive soon enough. At the moment it appears we are stuck with the classic peer-review system because, much like democracy, it may not always be pretty but what else is there?
posted by Sepp Hasslberger on Thursday January 29 2004
updated on Wednesday February 29 2012
URL of this article:
Alarming expose of research manipulation
Dr. Weldon's letter (at end) should make everyone question what they are 'officially' told The Autism connection is amply demonstrated when autistic children are cured though Mercury detoxification see: Mercury and Autism: Defeat Autism Now! (DAN!): Mercury Detoxification Consensus Group Position Paper Autism: a Novel Form of Mercury Poisoning [Short Version] [Long Version] Vaccinations Contributing to Rise in Autism? [Transcript] Aired 11/20/02 on CNNChris Gupta>... [read more]
November 15, 2003 - Chris Gupta
Science Commons: Open The Flow Of Scientific Information
Creative Commons has become a modern-day alternative to copyright which we inherited from pre-computer and pre-internet times. Ubiquitous copyright has become an obstacle to sharing and utilizing the immense amounts of information now at our fingertips. The idea is to form a "commons", that is, a freely accessible body of facts and ideas where barriers to the exchange and use of information have been removed or at least lowered from... [read more]
February 21, 2005 - Sepp Hasslberger
Gene Mallove: Science Censorship is 'Invisible Evil'
When in February this year, the Union of Concerned Scientists came out with a warning that "the Bush administration had systematically distorted scientific fact in the service of policy goals on the environment, health, biomedical research and nuclear weaponry at home and abroad", a long festering wound was touched, but unfortunately no cleansing process seems to be underway as yet. Examples for the distortion of science for purposes of either... [read more]
April 30, 2004 - Sepp Hasslberger
The spooky reality behind drug-related "news"
Tricks of the trade... ..."The long and short of the process is this: A drug maker funnels a highly-educated ghostwriter only the most flattering evidence - much of it no doubt doctored (no pun intended) - regarding a certain patent medicine's miraculous effectiveness and utter lack of side effects. Instead of being immediately used as toilet paper, this information is then skillfully crafted by the highly paid and totally amoral... [read more]
October 09, 2003 - Chris Gupta
Schubert: 'Sound Science' Overrides Reality and Common Sense
Science in the service of politics? Yes, says the Union of Concerned Scientists, appalled over the hijacking of science by political expediency. According to an article in The Register, more than four thousand scientists signed the latest protest against the Bush administration's appalling bending of scientific fact to fit the political agenda. David Schubert, head of the Cellular Neurobiology Laboratory at the Salk Institute for Biological Studies in La Jolla,... [read more]
July 18, 2004 - Sepp Hasslberger
Medline 'Oversight' - Orthomolecular Journal Not Indexed
The Journal of Orthomolecular Medicine regularly publishes interesting articles, such as the one by Alan Gaby MD which discusses "safety limits" for vitamins and which I cited in a recent post titled "Risk Free Vitamins - How Safe is Safe Enough?". One would think that Medline, the major internet-based reference work for medical scientific literature should list the Journal of Orthomolecular Medicine as part of its database of public information.... [read more]
June 22, 2004 - Sepp Hasslberger