Bands like The Beatles, the Jimi Hendrix Experience, Led Zeppelin, and Queen have spawned many tribute bands that produce note-for-note faithful renditions of original tunes like Can't Buy Me Love, Hey Joe, Stairway to Heaven, and Bohemian Rhapsody. They even look the part, with wigs, generous amounts of exposed chest hair, spandex, and vintage guitars. No one will applaud these tribute bands for their creativity but a certain segment of fans will appreciate being given the opportunity to relive the 1960s or 1970s.
In psychological science, replications of original studies are rare—at least in the published literature. And this is not the only difference with tribute bands. Another difference is that replication attempts are hardly viewed as tributes. No one is going to redo someone else’s experiment just for sentimental reasons (there is no scientific equivalent of a J.R. Stroop or Hermann Ebbinghaus tribute band). So why do replication studies?
This appears to be the spirit in which the Reproducibility Project is operating. Its members are trying to assess how reproducible the results are that three prominent psychology journals published in 2008. I am very interested in seeing the results. Not because they will allow us to tell which journal “won” (the smart money is on the Journal of Experimental Psychology: Learning, Memory, and Cognition) but because, taken together, these results will point to ways in which we can do our science better: replace some cracked stones, reinforce some sections, make a few additions, and sometimes perhaps abandon work on an entire wing. Moreover, they might lead the way to developing better building techniques.
My approach to replication is simple. I am not going after a series of results I find hard to fathom (otherwise, I’d be trying to replicate studies showing that priming with “professor” leads to better performance on a general knowledge test) or research by people I don’t like (I am fond of everyone). I am also not going to try to replicate a random sample of studies some of which might be on topics I don’t really care about that much (come to think about it, there are more than I would like to admit). My approach is thematically oriented.
I am interested in language comprehension and specifically in the mental representations that are involved. Are they word-like or more perception-like? I want to know how firm the foundation is of the little wing (subtle Jimi Hendrix reference) in that my fellow researchers and I have been laboring at. I want to know this so that we can develop better theories and methods (and not to kneecap a fellow researcher).
Together with Diane Pecher I recently published a series of replication attempts along these lines. It represents a first attempt. In an upcoming blog, I will talk more about the how and why of this research. The point here is that we should be concerned about the building but not the builders, except for the very small minority of them who are charlatans (more about them in a later blog as well).