The global movement toward open science in research has been a boon for psychology in many ways. Open data allow researchers to verify findings and catch errors. Greater access to studies also improves effectiveness by promoting more research from the same data (Arza, V., & Fressoli, M., Information Services & Useopens in new window, Vol. 37, No. 4, 2017). This can help accelerate new scientific developments—something that has proven particularly advantageous during the coronavirus pandemic.
But when it comes to integrating open science in the lab, it can be hard for researchers to know where to start. Many questions will likely arise: What is the easiest first step? Is this going to take a lot of time and money? Am I going to mess up? “A lot of people say, ‘Yeah, we can get on board with these things in concept, but what does it mean to actually do it?’” said Brian Nosek, PhD, executive director of the Center for Open Science (COS). “It can feel very intimidating; there are so many new things. But it isn’t an all-or-nothing practice.”
Instead, Nosek encourages researchers to embrace incrementalism—to start with something easy, like listening to a podcast about open science or organizing files on an open science platform, and then to take small steps from there. “It’s just like how we analyze data: Start with a t-test and then move step-by-step,” said Nosek.
Matthew Makel, PhD, an associate research scientist at Johns Hopkins University whose work focuses on open science research methods, offers similar advice. “Do what you can, when you can,” Makel said. “It may be easier to try to focus on incorporating one new practice the next time you plan a project. Once that feels comfortable, try expanding further.”
To help those working to incorporate open science into their lab, Nosek, Makel, and other researchers who practice open science shared their ideas on how to start.
Learn more about open science
There are now tons of resources that make learning about open science easy. One option is to become a member of an open science journal club. These clubs offer students and staff a space to discuss open science and reproducibility, said Naseem Dillman-Hasso, a PhD student who specializes in environmental social sciences at The Ohio State University and managed a lab that practices open science. For graduate students or investigators who don’t have a journal club at their university or institution, the grassroots journal club initiative ReproducibiliTeaopens in new window offers advice on how to create one from the ground up. Started in 2018 at the University of Oxford in the United Kingdom, this organization has since spread to 138 institutions in 26 different countries.
Another option is to listen to podcasts such as Everything Hertzopens in new window and The Black Goatopens in new window, both of which cover topics like preprints (the full draft of a research paper that is shared publicly before or during peer review), the benefits of open data and code, and how to preregister your study on a registry. Print articles can also help you learn about open science. “Easing Into Open Science: A Guide for Graduate Students and Their Advisors,” by researchers at the University of Minnesota Twin Cities and Lancaster University in the United Kingdom, for example, offers suggestions on how to begin engaging in open science with eight steps, ranging from easy (project workflow) to difficult (registered report) (Kathawalla, U.K., et al., Collabra: Psychologyopens in new window, Vol. 7, No. 1, 2021).
Resources like these can lessen peoples’ fear about using open science and make it more approachable, said Dillman-Hasso. They can also help break down misconceptions about open science, such as the idea that it slows down science. “I do think it’s slower pre–data collection, but once you move past the planning stages, it can actually speed up the process,” he said.
Another big misconception among some researchers is that the current scientific process isn’t broken, so there is no need to change it. “Psychology is on the forefront of this whole replication crisis, but there are a lot of other examples of this happening in nonpsychology fields,” Dillman-Hasso said. “A lot of the problems in fields like marine science and others are boiling down to fraud or research misconduct allegations that could have partially been alleviated by open science practices and open sharing of data and materials.”
Use an open source framework
One option is Open Science Framework (OSF)opens in new window, a free, open platform that offers researchers, journals, and institutions a range of tools for storing and managing research projects, sharing posters or slides with a broader audience, submitting preprints, and preregistering studies. A flagship product of COS, this platform has significantly increased the openness and reproducibility of scientific research.
There is no one “right” way to use OSF, said psychologist Nick Fox, PhD, a researcher at COS. “It’s very flexible and you can use it to store [data] however you want.” For example, when Fox starts a new research project, he takes the time to organize his project workflow by arranging his files and setting editorial permission levels for team members versus other users. Users can also link to third-party tools such as Dropbox or Google Drive and create new components or subprojects that form a hierarchy within the top-level project.
Another great feature of OSF is the ability to create a project directly from a template, which can keep your work structured and increase consistency across different projects, said Katie Corker, PhD, an associate professor of psychology at Grand Valley State University in Michigan who is an ambassador for COS and executive officer of the Society for the Improvement of Psychological Science. “Once you begin adding these different components, these different pieces—you might have your data, your materials, the manuscript, maybe a video demonstration of someone running through the protocol—you want to make sure that the information is organized not just for you but for any viewer or reader who is going to be consulting those materials,” Corker said.
All of this can save researchers a lot of time, said Corker. And OSF isn’t the only platform out there; other sites like GitHub and Zenodo offer similar features and tools.
Submit preprints
Another easy way to engage in open science is to submit a preprint. Submitting your preprint to services like OSF Preprintsopens in new window, PLOSopens in new window, or PsyArXivopens in new window, a free preprint platform for psychological studies developed by OSF, is a very manageable first step, said Nosek. “Posting a preprint takes about 5 minutes and is super easy to do.” This allows other scientists to give constructive feedback that will ultimately improve the piece.
Articles that are accepted during the peer-review process can also be shared on repositories, making the task accessible to more scientists, said psychologist Ummul-Kiram Kathawalla, a postdoc at the Edith Nourse Rogers Memorial Veterans Hospital in Bedford, Massachusetts. Earlier in Kathawalla’s career, she was nervous about submitting a preprint before it was accepted to a journal. “Now as I’m getting more comfortable, there are more things that I’m willing to post earlier on and get that feedback, which is really helpful,” said Kathawalla, first author on “Easing Into Open Science.”
Scientists tend to worry that posting a preprint will lead to a bombardment of criticism because far more scientists have access to the work before it’s been peer-reviewed. However, Nosek said, “rarely do those things actually happen. The criticism is not much different than publishing a paper.”
Share your data and code
Make a de-identified data set and post it on OSF or another data repository so researchers can download and use it. You can also create reproducible code by making a detailed version of the code that would allow another researcher to generate the same output. Both steps are key to facilitating collaboration across research teams, according to “Easing Into Open Science.”
For sensitive data that you don’t want to share or cannot share, you can set restrictions on who can see your data on many sites, including OSF. “There are plenty of research fields where the open sharing of data doesn’t make sense,” said Fox. There are more vulnerable populations that risk exploitation if their data are shared, for example, along with sensitive data that contain private or identifying information. “A push toward openness doesn’t mean trampling on those populations or those research fields that require more privacy,” he said.
Makel also makes sure he maintains all appropriate data protections, complies with institutional review board regulations, and protects participants whenever he shares data. Sometimes, data sharing isn’t possible—like when Makel uses secondary administrative data from schools that he didn’t collect. But other times it’s a great way to help your fellow researchers. “I recently had a meta-analyst ask me for something that we had not calculated in our paper. But we had shared the data,” Makel said. “Rather than go back through and run the new analysis myself, I just shared the link to our data.”
Just like with submitting a preprint, OSF has clear guidelines on how to share your dataopens in new window and codeopens in new window, making the processes straightforward for newcomers and experts alike.
Submit a registered report
Preregistration is the practice of specifying your research plan in advance of your study by writing out your hypothesis, what methods will be used, and how you will analyze the data. This information is then submitted to a platform like OSF and can evolve over time, if you are transparent as to why the study plan is changing, said Fox. “Preregistration is a plan and not a prison,” he said. Preregistration not only allows researchers to thoroughly think through the study process before collecting data but also guarantees clarity among collaborators, helps reduce unintentional false positives, and gives investigators the chance to get feedback from other researchers early onopens in new window. The result? A stronger study.
A registered report involves the same steps, but this information is then submitted for peer review by a journal before data collection—making it perhaps the most daunting aspect of open science. If accepted, the journal typically guarantees that they will publish the article, regardless of the results, as long as the researchers follow the established plan, according to “Easing Into Open Science.” Once the study is completed, the results and discussion are then added to the original document and the project receives another review.
All of this takes time to do properly, said Dillman-Hasso. “You have to spend all this time thinking and writing and have very clear hypotheses and contingency plans for what happens if the results that you’re expecting don’t happen,” he said. Additionally, “most journals don’t accept a registered report,” although the number of journals that do is growing quickly, according to a report in Natureopens in new window (Chambers, C., Vol. 573, 2019).
But Dillman-Hasso is still a proponent of registered reports. “It can often seem like doing things this way is slower, which is difficult, especially when there is this ‘publish or perish’ mentality,” he said. “In the long run, it will save you more time because you’ve thought through these decisions, you’ve made a plan. It will be easier on the outset once you start the data collection.” It also greatly reduces publication bias and an excess of positive results, which has been a problem in psychology (Scheel, A. M., et al., Advances in Methods and Practices in Psychological Scienceopens in new window, Vol. 4, No. 2, 2021).
Overall, much of incorporating open science into your lab involves creating and maintaining new research habits. Pretty soon, these habits will become as second nature as running an ANOVA, said Nosek. And jumping into open science now is far easier than it was 10 years ago when there were significantly fewer resources and less support, he said. “Early data suggest that the present enthusiasm for open science practices, particularly in psychology, is very high,” Nosek said. “The community has embraced open science to such a strong degree that it’s becoming the norm.”
