As a student, you may have heard buzz about the replication crisis in psychology. Some of the more sensational headlines paint a bleak picture of modern research and depending on to whom you speak, individuals can be optimistic while others are demoralized. So what is going on and is it really a crisis?
The dialogue around replication ignited in 2015 when Brian Nosek’s lab reported that after replicating 100 studies from three psychology journals, researchers were unable to reproduce a large portion of findings. This report was controversial because it called into question the validity of research shared in academic journals. Publication in high profile journals requires the research to be subjected to a rigorous peer-review process. At this point, it is assumed the conclusions shared are trustworthy and others can now replicate or build upon the work. Following the Nosek study, more labs began to conduct replications and a disturbing trend emerged: a large portion of studies across multiple disciplines in science failed the replication test.
Replication is vital to psychology because studying human behavior is messy. There are numerous extraneous variables that can result in bias if researchers are not vigilant. Replication helps verify that the presence of a behavior at one point in time is not due to chance. The report that the Open Science Collaboration (2015) put forth did not undermine the peer-review process per se; rather, it highlighted a problem within the research culture. Journals were more likely to publish innovative studies over replication studies. Following the trends of the journals, researchers who require publications in order to advance their careers are unlikely to conduct a replication. As a result, without continued investigation, the exploratory studies can be treated as established lines of research rather than fledgling inquiries.
In response to the replication crisis, more individuals have been embracing the movement of transparency in research. The Open Science Foundation (OSF) and the Society for Improving Psychological Science (SIPS) have created opportunities for researchers to brainstorm means of strengthening research practices and provide avenues to share replication results. Based on these changes, I would argue the issue of replication was not a crisis, but an awakening for researchers who had become complacent to the consequences of the toxic elements of the research culture. Highlighting the issue resulted in dialogue and change. It is a perfect example of the dynamic nature of science and captures the essence of how a career in research can be intellectually stimulating, rewarding and sometimes frustrating.
From a student perspective, engaging in replication research is a useful tool to develop your own research skills. I have found that many students have misconceptions about how to conduct research. Some common behaviors include:
- Assuming their idea is unique, but not conducting a thorough literature search to determine what is established.
- Proposing studies that are too complicated or have design faults.
- Lack of awareness of ethics or the approval process needed to conduct experiments.
- Lack of experience in regard to data entry or statistical analysis.
- Desire to change practices mid-study to increase participant compliance.
The replication movement has presented a unique opportunity for undergraduate researchers to provide meaningful contributions to science by bolstering evidence needed to substantiate exploratory findings. I teach a research seminar at Central Oregon Community College that requires teams to complete a replication study provided by the Collaborative Replication and Education Project (associated with OSF). Reviewers give feedback before and after data collection identifying problematic areas and insuring the study is an appropriate replication. Completed projects are shared on the website and exemplary studies are eligible for research badges. The process of replication requires students to slow down and analyze strategies. Over the course of the term, the student understanding of the process matures as groups question the choices of the original researchers. It is a high impact, low-risk educational environment because students learn valuable lessons whether or not the replication is successful.
Replication studies may not offer rewards for professionals, but there are direct incentives for students. Former seminar students have been able to add research experience to resumes, which, in turn, has allowed them to secure competitive positions in labs upon transfer to four-year institutions. My students have also reported feeling more prepared for upper division courses and more confidence in their abilities to conduct individual research.
If you would like to learn more about the replication movement or how you can begin a replication study, I suggest beginning with the reference below and visiting some of the websites of the organizations listed in this article.
Resources
Website for the Open Science Foundationopens in new window
Website for the Society for the Improvement of Psychological Scienceopens in new window
References
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. DOI: 10.1126/science.aac4716
About the author
Andria Woodell, PhD, is a professor at Central Oregon Community College in Bend, Oregon.
She teaches a variety of courses ranging from introductory psychology, development, social, positive and psychology of violence. She provides trainings to local and regional groups on psychological issues that arise in the workplace. She co-developed the COCC Teaching and Learning Center and is the current coordinator of the COCC Teaching Externship.
Beyond the classroom, her primary focus is preparing community college students for a successful transition into the university system. She co-advises the COCC collaborative learning program, which develops students’ professional skills and provides research/leadership opportunities beyond the classroom. In 2017, she earned the COCC Faculty Achievement award which is given to a faculty member who has demonstrated significant achievement in classroom teaching, leadership and professional excellence.
