Peer review is often considered a gold standard for scientific credibility. Do some experiments; figure out what is going on; write it up, checking other published work and covering all conceived possibilities, and submit it for publication. If the editors think it is worthy they ok it and it goes into the peer review process. Somewhere from 2-5 scientists with knowledge in the field (who may or may not have been suggested by the submitter) receive the document and read it through (or have a grad student/post doc read it). They check for scientific accuracy, spelling and grammar, cited works, and make a value judgement as to its importance/relevence relative to the journal in which it is being submitted for publication. Then they make a recommendation which can be: reject, a softer rejection with a suggestion as to where to submit next, accept with revisions, or accept as is (uncommon, in part because noting even a small correction is often seen as an indication that the manuscript was actually read).
The number of cycles and journals before acceptence depends on many factors including who is the principle investigator (PI), what is the subject, how much is new and/or unexpected, how well it is written, and who did the reviewing. Now most of you should have caught a couple important issues there: who is the PI and who reviewed it. Those are two factors that have very little to do with the scientific credibility and importance of the work. Yes, in general, when a famous well published scientist sends something out, it is likely to be quality. Yes, reviewers will have differing knowledge that will lead to them accepting/rejecting a paper for different reasons. Research science is competitive, however, and publications are the way to get grants, which are needed to get tenure, which is needed to be able to remain employed (in academia). This is likely to result in at least a few problems.
The whole of the peer review process presupposes that the submitted manuscripts are the result of honest, quality work. That the results are reproducable. That error and varience have been dealt with. The reviewers nor editors police this. When a paper is reviewed the reviewers are checking for importance and relavance first. Determination of factual issues and citations is second (some reviewers place this first). They read for mistakes--gramatical, graphics, et cetera--third. Ethics concerns do not enter. They are too hard to check for and there are too many submissions to deal with all of them. They are assumed.
If things are getting worse then it is likely because competition is forcing it--specifically competition for funding--not because the moral fabric of society is failing. Labs may rush things to beat others. They may take the first experiment that proves their hypothesis and run with it without checking for reproducibility. The best fix would be: increase funding. This is at the national level. NSF in particular needs the money. Bush promised it in his state of the union, but considering everthing he has said and done there is no reason to believe he will make good. Clinton did double the NIH research budget while he was in office, but their budget has stagnated making NIH grants harder to come by. We need a congress that will fund science, not only to increase and improve scientific breakthroughs/innovations in this country, but to ensure that scientists don't fall into an ethics or funding trap.
No comments:
Post a Comment