A growing number of psychologists are supporting greater use of real-world data and evidence—patient information derived from medical and insurance records, surveys, qualitative interviews, patient reports from home, and other sources outside of controlled clinical trials—to better understand and treat people, including those who tend to be absent from or underrepresented in conventional research. Among the traditionally underrepresented are individuals living in poverty, people of color, homeless individuals, and undocumented immigrants, who tend to fall off the radar of researchers, practitioners, and health care systems.
Those who use this data say it could be key to explaining why such groups are underrepresented and what treatments may or may not work for them as well as inform new ways to engage with them. In clinical practice, meanwhile, such data may help narrow what many psychologists say are serious gaps in knowledge about what happens when a patient leaves the office or when an intervention leaves the lab.
“Look at the many studies that have shown compliance with antidepressants and other pharmacological agents,” said Rossi Hassad, PhD, MPH, a behavioral epidemiologist and psychology professor at Mercy College in Dobbs Ferry, New York. “The studies are beautifully done—they show compliance—yet patients do not comply. When people go home and report back on how they did with the medication, they’ll say, ‘The timing wasn’t right; I didn’t feel good about it; it wasn’t convenient.’ You can’t capture that information in a controlled setting.”
Advocates, however, acknowledge that real-world data and evidence have significant limitations. Studies based on these data, or the evidence that they generate, can never fully replace the internal validity of randomized controlled trials, which through rigorous controls and standards attempt to establish cause-and-effect links between treatments and outcomes, said psychologist David Bard, PhD, a professor and chief research information officer at the University of Oklahoma Health Sciences Center.
Despite their shortcomings, though, real-world data and evidence can help determine if and how an intervention works with real people in real contexts, said Bard. And depending on the size and reliability of the data collected, they may serve as an important complement to smaller controlled trials that aren’t as good at real-world generalizability, he said.
“Real-world data is not without its flaws,” said Bard. “But we all recognize that this is definitely going to be the future. And we’re hoping that over time, we can figure out ways to add levels of rigor that existed only for small trials and pass it on to much larger trials. In the virtual world that we live in now, that’s becoming more and more practical.”
What is real-world data?
There is no uniform definition for real-world data, so researchers sometimes lump the data into the amorphous category of “any data that isn’t gleaned from a controlled trial,” yet that diminishes the distinct contributions of different kinds of research. The concept is also similar to and closely aligned with other approaches, including but not limited to implementation science, translational research, participatory research, precision medicine, qualitative research, and mixed-methods research.
That said, real-world data is often associated with “big data,” which currently means large-scale data analyzed through advanced means and then used to target or predict behavior. Thanks to these expanded analytic techniques, researchers can now cull new and nuanced real-world information from electronic health records, insurance claims and billing activities, product and disease registries, even wearable devices (see Wearable devices as therapy tools). In 2016, Congress gave a green light to this methodology with the 21st Century Cures Act, which highlights the potential for these tools to determine risks and benefits of new pharmaceuticals and help regulate those products as result. In the mental health arena, such data could eventually provide population-level information regarding variations in emotions, thoughts, and behaviors across time and contexts, Rossi noted.
In the realm of traditional clinical science, Bard sees two main ways that psychologists can use real-world data and real-world evidence, or findings derived from that data. One is trolling large data sets to find new topics or trends to study. For example, big data may reveal that a subset of people reacts more poorly to one given treatment than another, and researchers can then create hypotheses based on those findings.
Another way to use the data is to test existing treatments under real-world conditions. Called “pragmatic trials,” these are a “more intentional look at the effectiveness of a program, with some additional rigor involved,” Bard explained. “They aren’t completely exploratory, but they do relinquish some controls that we would traditionally use in the lab.”
For example, instead of running a study in which researchers use specially trained providers to deliver an evidence-based intervention in the lab, they might instead tap providers who work in community clinics or private practice to deliver the intervention. Or investigators might supplement primary outcome data with administrative data already being collected by private practices or mental health agencies, Bard explained.
Other psychologists envision using real-world data more broadly, such as by using qualitative patient data to inform treatments or interventions. Examples include designing interventions based on data you’ve collected on a community you’re studying, using homework data from clients to guide treatment, or using social media trends to target therapy or intervention directions.
“I think of real-world data as observational data, as what we are experiencing in the field,” said Jessica L. Jackson, PhD, a counseling psychologist in Houston and clinical care lead at Modern Health, a mental health technology firm that links consumers to data-based mental health services and technology support tools. “Our ability to use it is something clinicians tend to discount because we don’t always consider ourselves researchers.”
Other providers agree that it’s a matter of being creative. Common themes you hear from patients, such as multiple requests for a certain type of service or numerous appeals for specific information via your social media account, are real-world data that can be leveraged in multiple ways.
“When you have data that are broad in scope, you have to approach them with a mindset of discovery,” said Hassad. “If everything is hypothetical and deductive in our reasoning”—as is the case with controlled trials—“then there is no room for creativity and discovery.”
How is real-world data being used?
While the significance, value, and definition of real-world data are still under debate, the field is blossoming with examples of the methodology’s potential power, in both quantitative and qualitative ways.
The San Francisco Department of Public Health’s behavioral health team, for example, has been using real-world data on a broad scale to pinpoint underserved individuals and communities in the city and make sure they have better access to high-quality mental and behavioral health services. Through continuous data analysis and intervention, the department also aims to improve on those services.
Their work hinges on the collaboration of an interdisciplinary set of mental health researchers, epidemiologists, data analysts, and other data experts. They deploy sophisticated, systemwide tracking and analysis of mental health data from multiple public health sources—including health and behavioral health records, local systems of health and human services, and program evaluation interviews—to determine whether programs are meeting the mental health and support-system needs of community residents or where gaps still exist. The team also relies on strong relationships with federal, state, and community partners to support and facilitate their work, said Seth Pardo, PhD, a behavioral epidemiologist on the team.
A signature program is Gender Health SF, which promotes access to medical care for uninsured and underinsured transgender people, including undocumented immigrants. Thanks to the team’s data-driven approach, the program now provides evidence-based wraparound services for those considering gender-affirming surgery. A central feature is a cadre of peer supporters who walk patients through every step of the process, connecting them with medical and mental health professionals and resources and providing emotional support, explained Pardo.
The team’s analysis and curation of real-world program data—which are showing large improvements in patients’ physical health, mental health, and functioning—made it possible to expand and ultimately gain full insurance reimbursement for the program, Pardo said. Yet its true benefits can’t be captured in numbers, he said.
“We hear people in these communities say, ‘This program saved my life,’” he said. “We know that suicide rates are really high in this community, and we have had very few in our population.”
Others are creating specialized real-world data registries that are relevant to psychologists’ work. Researchers at Morehouse School of Medicine in Atlanta are partnering with Trulieve GA Inc., a medical cannabis company, to build the first real-world data and genomic data registry related to medical cannabisopens in new window use and its effects on pain management, mood, performance, health behaviors, and other factors. (As of July, a Georgia commission had announced an intent to award Trulieve GA Inc. a cannabis production license.)
The registry will gather comprehensive data on between 5,000 and 10,000 licensed medical cannabis users, including medical records, insurance claims and bills, surveys and questionnaires, mobile device data, and more. That information will be paired with genetic data showing differences in how individuals process various medical cannabinoidsopens in new window, said immunobiologist James W. Lillard Jr., PhD, a professor and associate dean for research affairs at Morehouse, who is leading this effort.
The goal of this research partnership is to inform the growing field of precision medicine, an important potential use of medical cannabis, said Lillard. “Once we have enough data, we’ll essentially be able to create personalized medical cannabis solutions,” he said. “So perhaps this will become an alternative to things like opiates to better manage pain.”
Importantly, this work uses a health-equity lens, Lillard added. “For precision medicine to be a reality for everybody, you need to have diverse patient populations in studies like this,” he said. “Because in many ways, we’re all the same, but we’re also very individual.”
Additionally, psychology has its own registry that may eventually contribute useful real-world data to the field, Bard said. Through its Mental and Behavioral Health Registry (MBHR), APA aims to recruit thousands of licensed psychology practitioners to use a unified measurement system to monitor patient care and outcomes, said Bard, a member of the registry’s advisory committee. This qualified clinical data registryopens in new window is tied to Medicare reimbursement so that providers who use the system and have positive outcomes receive higher reimbursement rates or incentives.
The endeavor—still in its early stages—underscores the challenges and promises of real-world data, Bard added. “It’s inevitable that these data will exist,” he said. “And then the question becomes: How accurate are the data? And what can we do to improve the reporting systems and the validity of the responses?”
“The hope,” he added, “is that these data will not only inform how we finance and evaluate the quality of the care but also allow us to probe deeper, more nuanced questions about what works for whom and under what context.”
The qualitative side of real-world data
Not every psychologist has the interest in or ability to analyze big data sets. Still, there are many ways that psychologists who are interested in more qualitative or observational approaches can tap into the power of real-world data to enhance their practices or studies, those involved say.
Psychologist Tali Ball, PhD, director of the Stanford Translational Anxiety Research Lab at Stanford University, for example, uses neuroimaging to study adult anxiety. But for a participatory research project that she is starting in the San Francisco Bay Area, she plans to collect real-world data and evidence in a more qualitative manner.
“We want to ask people in an open-minded way about their concerns and about the issues facing their community, and how our expertise might be helpful to them,” she said. “So, we’re starting with a small sample and a lot of depth.”
Clinical practice is also ripe with opportunities to use real-world data to help clients, said Jackson.
For example, during the coronavirus pandemic, Jackson and colleague Farzana Saleem, PhD, of Stanford University, launched Communicating Race Fullyopens in new window, a user-friendly website featuring infographics and videos to help parents talk with their children about racism. Jackson and Saleem relied on research and themes they collected from parents through their clinical practice to create the content.
Common themes that clinicians hear from patients are a type of real-world data that can help guide treatment and inform psychological interventions in multiple ways.
“Parents would ask things like, ‘At what age should I talk to my child about this?’ and, ‘How do I actually have the conversation?’” Jackson said. “It’s the kind of observational qualitative data that I think a lot of clinicians have that wouldn’t necessarily show up in a research data set.”
Meanwhile, at Modern Health, researchers take information they glean from the firm’s app to determine patient preferences, both on individual and collective levels, Jackson said. For instance, an individual may express a preference for online mental health coaching rather than more intensive therapy, and the team can use that information to guide them to appropriate help. Or the data may reveal that many people want more digital offerings on anxiety, parenting, or other topics.
“Our intention is not, ‘Let’s do a research study,’” said Jackson. “Our intention is, ‘How are people using our app so that we can create resources and better pathways for mental health that support that?’”
While real-world data aren’t as neat and clean as those that emerge from controlled trials, they promise to provide a welcome and necessary balance to traditional research, added Hassad.
“Real-world data is messy data,” he said. “And because it’s messy, we want it, because it reflects the real context in which people live and operate.”
