speculation/testability Pt. II (peer review)

[Crpntr@ix.netcom.com (Kenneth Carpenter)]

Mickey Rowe wrote:

"However, it would concern *me* if it didn't concern *you* while you 
were reviewing papers....."

"However, if I personally were reviewing a paper, I'd be hesitant to 
recommend acceptance if the author(s) couldn't even suggest a method for 
testing the main premise..."

"In a current peer reviewed paper though..."

"As a reviewer, it may not be your job to suggest tests.  As an author 
it should be.  And as a reviewer it should probably be your job to 
recognize when an author hasn't done their job."

I would like to cite from "How to Write an Influential Review" by 
Michael L. Rosenzweig, Jerrold I. Davis and James H. Brown.  I stumbled 
upon it on the Internet, but I don't remember where (sorry).  
It is reproduced from Plant Science Bulletin 40(1):6-8 (1994), a reprint 
of Plant Science Bulletin 34(2):5-7 (1988).  

     "Surely a review should seek to identify and encourage the most 
promising and innovative research.  Yet it is precisely the newest ideas 
that are the least tested, the most controversial and easiest to 
criticize. In fact, history reveals that new theories often are 
incomplete and often do contain serious errors.  But wouldn't it have 
been a tragedy if Natural Selection had been rejected because Darwin 
founded it upon incorrect mechanisms of inheritance.
     "If we are to advance our science, it is necessary that we take 
risks and actively encourage the development of new concepts, theories, 
and methods.  If we as reviewers are afraid of mistakes, and insist that 
our peers write airtight proposals, then who will dare to tackle the 
difficult questions?  If we required that a proposal be so well 
described that we  can visualize every aspect of the research, is the 
work really likely to produce any surprises or major new discoveries?  
We have to be honest and point out potential problems, but, above all, 
we must strive to identify and express our enthusiasm for new ideas and 
innovative approaches...."

    "The more negative the tenor of a review, the more detail it has.  
This is a fact.  We know no reason why it has to be so, but it is.  
Negative reviews are often full of well-reasoned objections.  Positive 
reviews are more often brief statements of approval.  They may be full 
of glowing adjectives, but they rarely contain the details and logical 
arguments which would give them substance...."

    "Many reviewers seems to think it is their primary responsibility to 
discover and call attention to all the flaws in a proposal or 
manuscript. Perhaps they have the attitude that it should be deemed 
worthy until proven otherwise.  They may sound fair, but it is, in 
truth, pernicious. It makes us over into petty bookkeepers, subtracting 
the value of each counterfeit penny without noticing that they are 
coming from a solid  gold box.  If we adopt the attitude that a proposal 
does not deserve funding unless the research is daring, novel or 
interesting, then we should place more emphasis on the positive aspects 
of a good proposal and write longer, more constructive positive reviews.
    "We need to remember that it is much more damaging to our discipline 
to suppress an important contribution than to fund or publish a 
questionable piece of research.  New ideas and conflicting data cannot 
have any influence unless they are developed, whereas serious errors 
will usually be detected and corrected, either by the investigator 
before publication or by the scientific community soon afterward.  This 
is why it is essential to be broad-minded, and to consider the potential 
importance of a piece of research as well as to search for flaws. "

   "The above is not intended to suggest that we should endeavor to be 
less critical.  Serious criticisms and substantial concerns should 
always be expressed, but this can be done by dispassionate language 
without indulging in _ad hominem_ assaults.
   "Critics of art and literature, whose criticisms are often published 
and who earn their keep from them, feel they must entertain their 
readers with a rapier wit, caustic comments, piercing put-downs, and 
acid cuts. Many appear to have decided that criticism is a written 
version of prize fighting, except that in boxing, low blows are against 
the rules.  Leon Wieseltier calls it "aggression as an intellectual 
instrument".
    "Unfortunately, all too many negative scientific reviews seem to 
have been written by put-down artists.  This is not only cruel and 
cowardly (at least the literary critic signs his piece), but it 
minimizes their influence as well.  The editor, panelist or program 
director is driven to sympathize with the victim.  This may mean that if 
you and you alone noticed the flaws, but reported them intemperately, 
your criticism will be ignored.  Moreover, such aggressive attacks leave 
a lasting impression of unprofessionalism on your part."

    "...Dr. Janet V. Dorigan of the Department of Energy has observed 
that when scientists are under attack, they circle 'round, wagon train 
style. The physicists aim outward at their opponents.  Biologists, on 
the other hand, aim inward, at each other.  Their weapons, of course, 
are disparaging reviews and negative comments...."

    "In order to convey a more accurate impression of the view of our  
collective labors, we all need to make a conscious effort to tolerate 
diverse ideas and unconventional approaches, and to promote independence 
and originality.  Robert Reich has written that "Technological 
innovation is largely a process of imagining radical alternatives to 
what is currently accepted." In our reviews, we must encourage that 
dissent and emphasize the advances it will make possible..."