Scientific Repetition

Reading through this Vox article I nodded quite a bit.  A few quick notes...

In graduate school I did some research that...didn't discredit so much as countered previously published work by another group (note: I seriously doubt it was fraud).  Nothing came of it.  In a perfect science world, it would have been written up.  Not a major publication, but a short note, with the relevant data and the [correct] explanation.  Except: most researcher's wouldn't really care, lots that would may have figured it out anyway (actually, probably not, but I like to think the best of other scientists) and publication of a short note like that, which really only exists to say "those guys were wrong" seems...petty and spiteful.  That I didn't know the other researchers at all is irrelevant, petty and spiteful are not good reasons to publish.

Still, it would have been best for that particular field of science had that work been published.  Better information which may have helped other researchers doing similar work and referencing the older, [partly] incorrect work.  But had we tried, I've no idea where it would have been accepted.  It's another side of the coin from a "failed experiments" journal.  Science that should have worked but didn't, things that did work, but for which the explanation was wrong or incomplete.

There's a lot of science out there and so there are a lot of these "failures".

One point that I would like to make has to do with the statement in the article that

Most studies aren't replicated — and researchers are discouraged from doing so

which may be true for some longer/larger experiments.  Those that require a full grad student career (or more) to complete, or that are so expensive, that repeating them practically requires a grant application that details "why we should do this again".  In many of those cases, lots of researchers are involved early on and do a lot of checking of each other's work.  Also, in many cases, the raw data is available for subsequent researchers to look into.  But for a lot of smaller experiments--which actually make up the bulk of science research--the studies often are replicated, in the same lab by subsequent students, or by other labs with similar interests/directions.

In those cases, problems do get found out and understood and passed around, but very rarely does that occur via publication that directly counters previous work.  More likely a post-doc comes into a lab knowing about the old paper, and a senior level grad student informs her "actually, that wasn't right and this is what really is happening" and the post-doc is now informed and does research with her new understanding...that has not been published.  Sometimes you can comb through a trail of publications and note that a change took place without any acknowledgement to it taking place, and sometimes you have to either go through the lab or know people connected to it to find out.  So researchers connected to a research network may know a particular thing doesn't work, or didn't mean what it was initially thought, but researchers outside that network may not.