The scientific method is the cornerstone of psychological research. Knowledge is systematically pursued, beginning with the recognition of a problem or phenomenon of interest. The situation is studied; hypotheses are formed and then tested through experimentation. Once the data are in, hypotheses are confirmed or modified on the basis of the information and the conclusions are drawn, but there is still a final step to the method: replication.
This final step is sometimes overlooked or forgotten, as it's not quite as exciting as a new line of inquiry. Yet replication is an essential part of the scientific process; without it, it's hard to tell whether the first test of the hypothesis was done well enough to justify conclusions, what the limits of the hypothesis might be, whether there were sampling errors, whether artifacts influenced the results, or whether the conclusions can be generalized to a broader population than the one specifically tested.
The February 2014 issue of Psychology of Aesthetics, Creativity, and the Arts features a special section, "Replications in Psychology," that focuses attention on this often overlooked part of thorough scientific inquiry.
In the lead article of the special section, "The Empirical March: Making Science Better at Self-Correction," Matthew C. Makel shines a light on replication and other research quality issues in psychology. Replication has the potential to improve the quality of psychological research, but it has a few hurdles to cross first at the level of the individual researcher.
For example, Makel points out a bias of journal editors against publishing replications, because of the belief that such articles add little information to the literature and take up journal space that could be devoted to new ideas. It's hard to tell how many replication articles have been rejected or never submitted to journals, given the emphasis on novelty in the social sciences.
This emphasis hobbles the self-correction expected in science, which is achieved through, among other methods, replication studies. Questionable research practices (e.g., hypothesizing after results are known [HARKing], selective reporting) and acceptance of underpowered manuscripts also add to the research quality problem.
Makel proposes five categories of change to fix the detailed problems:
- changing the focus of articles from significance to facts
- making actual predictions rather than HARKing
- revamping the review process
- adjusting the publication process
- running more replication studies
Matthew T. McBee and Michael S. Matthews, editors of the Journal of Advanced Academics, provide the first response in "Change Starts With Journal Editors: In Response to Makel (2014)." They discuss three of Makel's focal issues — the "file drawer" problem, a dearth of replication studies, and inadequacies in null hypothesis testing — and argue that journal editors can mitigate these damaging practices.
Dean Keith Simonton, in "Significant Samples — Not Significance Tests! The Often Overlooked Solution to the Replication Problem," argues that choosing significant samples — a collection of exemplary cases of a phenomenon rather than a group intended to represent and generalize to a large population of generic human beings — negates the need for significance testing and permits exact replication.
In "Pipeline Revisions: A Call to Change," Roni Reiter-Palmon and Pablo P. L. Tinio, the editors of Psychology of Aesthetics, Creativity, and the Arts, provide an editor's-eye view of the conundrum of judging the significance of the replication article submitted for publication, as well as the ramifications of the resulting decision. They note that reviewer guidelines need to change for replications to gain a greater share of journal pages.
The editor of Empirical Studies of Aesthetics, Oshin Vartanian, in "Toward a Cumulative Psychological Science of Aesthetics, Creativity, and the Arts," discusses how reaffirming the importance of general theory building and improving researchers' ability to identify important new lines of inquiry will help make the psychological study of aesthetics, creativity, and the arts more self-correcting.
In "Toward Avoiding an Empirical March to Nowhere," Ronald A. Beghetto, editor of the Journal of Creative Behavior (the oldest journal dedicated to creativity research), advocates taking a step back to make sure that theories are strong and methods are solid before testing and publishing about new and replicated work.
Jeffrey K. Smith and Lisa F. Smith take issue with the idea that lack of published replication studies is a problem in psychology today; they provide their own suggestions for invigorating the field.
Sandra W. Russ, in "Demonstrating Effects With Children: When Do We Know We Know?" notes that children complicate creativity research in myriad ways, as they do everything else they touch. As a result, replication studies take on extra importance in this subfield of psychology, even though such replication studies are more involved to plan and conduct.
Finally, Matthew C. Makel and Jonathan A. Plucker respond to the comments in their rejoinder, "Creativity Is More Than Novelty: Reconsidering Replication as a Creativity Act." They also reiterate Makel's call for replication studies and illustrate why replications are an important part of any research field truly interested in science and innovation.
Citation:
Plucker, J. A. (Ed.). (2014). Replications in psychology [Special section]. Psychology of Aesthetics, Creativity, and the Arts, 8, 2–29.
Note: This article is in the Social Psychology & Social Processes topic area. View more articles in the Social Psychology & Social Processes topic area.