Speaking of Psychology: How to choose effective, science-based mental health apps, with Stephen Schueller, PhD

Among the thousands of apps that aim to help people with everything from stress to anxiety to PTSD to sleep problems, how many are based on solid scientific research? How many live up to what they promise? And how can you choose from among all the options?

Stephen Schueller, PhD, is an associate professor of psychological science and informatics at the University of California, Irvine and executive director of One Mind Psyberguide, a mental health app-reviewing website. His research focuses on improving the accessibility and availability of mental health resources through technology.

Kim Mills: 2020 has been a watershed year in digital mental health care. The coronavirus pandemic and a summer of racial reckoning have left many Americans stressed, anxious, and depressed, while at the same time, the pandemic has made it more difficult to obtain in-person mental health care. Can technology help to bridge that gap? Many consumers seem to think so. One market research firm found the downloads of the top 20 mental wellness apps rose almost 30% from January to April this year. But among the thousands of apps that aim to help people with everything from stress to anxiety to PTSD to sleep problems, how many are based on solid scientific research? How many live up to what they promise? And how can you as a consumer make informed choices based on your mental health needs?

Mills: Welcome to Speaking of Psychology, the flagship podcast of the American Psychological Association that explores the connections between psychological science and everyday life. I'm Kim Mills.

Mills: Our guest today is Dr. Stephen Schueller, executive director of One Mind PsyberGuide, a nonprofit mental health app reviewing website that's been called the consumer reports of digital mental health. He is also a professor in the Department of Social Ecology at the University of California, Irvine, where he works on developing and evaluating digital mental health technologies with the goal of expanding access to mental health care.

Mills: Welcome to Speaking of Psychology, Dr. Schueller.

Dr. Stephen Schueller: Thanks so much. I'm really excited to be on here.

Mills: As I just mentioned in the intro One Mind PsyberGuide has been called the consumer reports of digital mental health. Why did it get that moniker? What does that mean? And is this something that we need out there in the marketplace?

Schueller: I definitely think it's something we need out there in the marketplace. There are a host of different digital mental health products, most of these being apps. If you were to go to the app store and to put into the search box depression, anxiety, stress, PTSD, you would get tens of thousands of responses across those different terms. In fact, some estimates suggest that there's about 20 to 25,000 mental health apps that are out there. And trying to find a good one is like finding a needle in a haystack. And so definitely we find that a lot of consumers do need some help in terms of separating the good from the bad, the effective from the not effective, the science backed from the ones that aren't science backed.

Schueller: And what we do at One Mind PsyberGuide is we identify and we review products on multiple dimensions. We look at how credible is it? Does it have science backing that this product actually works? Is it based off of evidence-based techniques? We look at the user experience. Is it easy to learn, easy to use, easy to navigate? Is it free from technical glitches? And we look at issues around data security and privacy. Does the app live up to standards of regulation? Does it keep your data safe? Does it not sell your data to third party individuals? And I think that all these things are that should figure into a decision on whether an app might be a reasonable product for you to try to be able to promote your own mental health and wellness.

Mills: How do you at the PsyberGuide decide which apps to review since there are so many out there covering such a range of conditions?

Schueller: That's a good question, one we get asked a lot. It's a little bit of an art and a little bit of a science. And so we really want to focus on products that people are using. We look at the number of downloads and the use of those products by consumers. We want to focus on products that are backed by science. And so we constantly are looking at the scientific literature in terms of what works, what has evidence behind it. We also want to focus on things that are useful for people. And so we focus on products that fit the needs of different consumers. We have partnerships with various organizations and they help us figure out what products their constituents are interested in.

Schueller: And so I think the processes that we use to identify what products that we look at kind of aligns with the ways that people decide to use these things, that they hear about them from trusted sources, from their doctors, their friends, from family members. They read articles about them on social media or in websites, or they might see an advertisement for a product in a couple of different places. And so I think we, at One Mind PsyberGuide, we're constantly looking all that information. And then we prioritize things based off of, again, what people are using and what the science is saying.

Mills: Who are your reviewers? And what are their qualifications? How do they become reviewers for the PsyberGuide?

Schueller: At One Mind PsyberGuide, we have a team of different reviewers in house that review different products for our website. And those reviewers all undergo training to be able to use the materials we use to be able to evaluate products.

Schueller: We also do some contracting out for some of our review material to outside experts, to clinical psychologists and other mental health professionals. But those are usually what we call professional reviews. And so on the One Mind PsyberGuide website, in addition to those criteria that I mentioned, the credibility, the user experience and transparency around data security and privacy, we also have sort of a narrative description of how you might use this product in your life and what are the pros and cons and what are some of the different clinical considerations. And those narrative reviews are all written by professionals. And so the individuals doing those credibility user experience and transparency reviews are all members of the One Mind PsyberGuide team that are trained to be able to use those rating scales with fidelity and reliability.

Mills: Do you collect any information from the users and use that as a basis for how you review various products?

Schueller: That's a good question. I think it's something that would be really useful. It's not that we actually currently do at One Mind PsyberGuide, but definitely something that we have a lot of interest in. We've done a little bit of that information before, but it's hard to track that. And I think that's actually one thing that really necessitates the... having a product like One Mind PsyberGuide is that you would think about where those consumer reviews often live and those are in the app stores. And so you go to the app store and you see this person thinks this is a five star app, this person thinks this is a forest startup. And what we find is that those ratings and those user ratings from the app stores often don't relate to the credibility and the benefits of those products.

Schueller: Now, I think there's some problems with the way that those ratings appear in the app store. I think that there's a lot of companies that are probably paying people to put positive ratings in the app store or they're biased, so only the people are having really good experiences are going to the app stores to leave those ratings. I don't mean that to say that I don't think a consumer... the impact on consumers and the benefits from consumers and the views of consumers are important. I just think it's a tricky thing to figure out how to collect systematically and in an unbiased fashion, like we try to do what the work we do at One Mind PsyberGuide, and it's something that we're still trying to figure out the best way to be able to incorporate. I would really hope that that would be something that One Mind PsyberGuide would be using in the future. But I think it's still a really important open question about how to best collect that information.

Mills: Yeah. Kind of risky, like using Yelp to choose the restaurant where you want to eat tonight.

Schueller: Well, and I think it's a great point. And I think, I sort of... I love restaurants, so I like the metaphor. I liked when we used to be able to go to them. But I think what we think about sometimes at PsyberGuide is that we sort of are the Michelin Star equivalent. We have experts going... we're trying to figure out at the best, what is this product capable of? And I think the Yelps are important as well, because there's a lot of information and value that gets added from consumer ratings and Michelin Stars and Yelp ratings don't always line up.

Schueller: And I think that we would need both, but I think we need both in a way that sort of acknowledges what are the pros and cons of each approach. If I have a four and a half star rated Taco Bell, I don't think about that as being on the same, you know, standard is a well-reviewed Mexican restaurant that has a Michelin Star, but maybe that tells me something about that particular location. And so I think bringing that information together, I think is really valuable, but again, it has to be done in a way that understands that each of those approaches, the Michelin approach and the Yelp approach have strengths and limitations, and you want to leverage the strengths while downplaying the limitations.

Mills: How much should consumers rely on apps for their mental health issues? Are there some conditions that are better suited to apps than others? For example, I can't envision someone with a serious mental health issue, such as say schizophrenia, getting much help from an app.

Schueller: Yeah. Well, first I'd like to say that apps are no replacement for therapist. These are not meant to be replacements for traditional mental health services or for mental health professionals. And so I think that there's sometimes a false dichotomy made between apps versus traditional treatment. And I think that really the best case scenario should be technology plus traditional services, technology and human services.

Schueller: When we look at the research, however though, pretty much anything you can treat through a psychosocial intervention through traditional treatments, you can treat using a digital technology as... either in some sort of combination with a human supporter or with some sort of relationship with some human support. And so there are definitely, there are apps out there for psychosis. There are apps out there for bipolar disorder. There are apps out there for a host of serious mental illnesses.

Schueller: I think actually, interestingly, I think one of the products that I'm really interested in when it comes to psychosis and schizophrenia is looking at these cognitive remediation products. These are technologies that essentially provide brain training to be able to improve the cognitive functioning and those tools work. There's a lot of robust, scientific evidence behind them. But again, those aren't seen as a replacement for some of the other pieces that are part of the treatment than an individual with psychosis or schizophrenia might receive to be able to promote their recovery. We find that these tools work for a variety of different people.

Schueller: Now, I think depending on the mental health challenge that someone's facing, there might be more or less appropriate uses of these technologies and sort of the continuum of care. For example, in the UK, the national Institute for clinical excellence actually recommends digital treatments as frontline treatments for the treatment of depression, anxiety. They don't have that same recommendation for serious mental illness. And so I think for common mental health issues, these might be very nice sort of frontline treatments, or might be ways to sort of really introduce the skills that get taught in traditional psychosocial treatments, but for more serious mental illnesses, they might play a slightly different role. I think research really supports that these things are useful across a variety of conditions, but the way that we use the might differ.

Mills: How much has this being taught right now in graduate psychology programs? Are people being trained to use apps as part of the therapy that they may offer in the future?

Schueller: I don't think it's a major part of current clinical programs, at least not broadly trained. I think that there's still a developing expertise in this area. I think there's some places where we see a lot of good work and training and implementation. For example, the Department of Defense and the Veterans Administration have been front runners, both in the development of these tools. They were leading experts in terms of developing some of the first mental health apps, including PTSD Coach, and they continue to have a suite of different products. And because they have so many different tools, they're really training people in the DoD and VA to use these tools in their practice.

Schueller: But I think if you go to graduate program in psychology, there's probably not a lot of work that's really looking at teaching people how to use technology thoughtfully in their clinical work. And so I think this is something that really would be an important area to address in sort of future training programs. And it's something that I care a lot about because I think that, again, this is not technology instead of people, and so I think we need to train a workforce that's educated to use these tools in their practice.

Mills: What makes for an engaging user experience? Why are some apps stickier than others and make users come back and doesn't that make them more effective if people actually use them systematically?

Schueller: Yeah. I think that is the case, that people who use these products more tend to get more out of them. I think that the science of engagement is a challenging area, because I think that there are reasons where people may not use a tool as much and still benefit. For example, there might be... the term I use is happy abandonment, which is I got what I needed out of this app and now I don't need to use it anymore. And that there's also some mechanism through which the people who need the most help tend to use the product the most or an app the most. And so there's a relationship between clinical severity and app use that sort of complicates understanding of the relationship between engagements and outcomes.

Schueller: That being said, the only way these things could work is if someone uses it. And so I think to answer your question about what makes these apps sticky or engaging or makes people want to come back, I think simplicity is a really sort of important piece that it has to be a product that's easy to use, that integrates into your life well. I think a lot of these sort of digital health technologies make people's lives harder by giving them something more to do, and none of us need something more to do in our lives. And so I think the best products are ones that sort of simply and... reduce burden and reduce complication by sort of seamlessly integrating to a person's life.

Schueller: I think also being able to meet the needs of the user. Some people might really want videos and other forms of media to engage in. Some people might do well from having very brief sort of exercises to read. And so I think it's also the case that not one app is going to meet the needs of sort of all different users or really appealed to everyone. And so I think understanding who the app is built for and having the content and the interaction styles and the things that really kind of fit that different person.

Schueller: I think an important implication of that frame, that thinking is that for a consumer, if you're interested in using one of these products, to try a couple of different of them, so if you want to use a mindfulness app to download maybe two or three mindfulness apps. Try out a couple of the meditations, see which ones really sort of fit your lifestyle the way you like to engage content, the ones that make sense to you, that you like the color schemes. All those things, I think, play a big role and then to decide to kind of use that product. And I think that's one of the benefits. I kind of introduce this as a challenge is there's 20 to 25,000 things out there, so it's really hard to find ones that are evidence-based. But I think the flip side of that is because there's so many, that means if you do a little bit of searching, you might be able to find one that really does fit what you need when you're looking for one of these products.

Mills: Let's talk a little bit more about data privacy. You mentioned that as being one of the aspects that you look at when you're evaluating apps, and there's been a lot of chatter in the news lately about at least one of these big app companies possibly mining user data for marketing purposes. How much should consumers be worried about this and what can they do to protect themselves?

Schueller: Yeah, I think it's definitely something consumers should be concerned about. And we did a review where we looked... we took a deep dive into the data security, privacy policies of about 120 different apps for depression. And we found that about half of those didn't have a data security and privacy policy and of the half that did, only half of those privacy and security policies were what we deemed acceptable. There's a lot of variance and there's also a lot of... or I should say there's not a lot of information out there for a lot of these products.

Schueller: I definitely think that a consumer should read the data security and privacy policy. And look, I know I looked at a lot of the terms and services and privacy policies for different technologies. I do, and it's a big wall of text and I just click accept and I move on. But I think one thing I think is important... one easy thing to sort of look at is does the product have a data security and privacy policy, because I mentioned many don't. And so I think that's a red flag in itself. If they don't tell you what they're doing with your data, you probably don't like it.

Schueller: As I mentioned, those Department of Defense and VA apps, I think those are very good apps with very good data security privacy policies, because they are developed by the VA and the DoD. And so they don't collect the type of information that some of these other products collect that are potentially using you as a revenue generating source when you're using their product. And so I think looking at where the app comes from is important. I think the other thing to consider is everyone is a little bit different in terms of their sense of what data is being collected from them and how comfortable they are with that. I fully appreciate that when I use Google Maps, Google knows where I'm going all the time and that's a trade off I'm willing to make because I'm not going anywhere that's all that exciting and I really like to use Google Maps.

Schueller: And so I think that if we think about data and privacy as somewhat transactional, that a person can be, hopefully, make an informed decision on whether the transaction, the benefit they get from that product is worth whatever they're giving up or potentially giving up when they're using that product. And that's one reason why we actually call our data security and privacy scale a transparency scale, because to us it's really... the important piece is the transparency of the information, such that a consumer can make an informed decision of whether... what information is being collected from them is worth the benefit they're receiving from using that product.

Mills: What are some of the best mental health apps out there for various conditions? I mean, I know there's no one size fits all answers, but there must be some that rise above the rest.

Schueller: Yeah. I mean, I think that it's a challenging question. And definitely one thing that we do at One Mind PsyberGuide is we don't endorse any specific product. We have our school [inaudible 00:19:25] and we sort of emphasize... what I've noted for you is that there's different ways to evaluate these things and there's different aspects that people might care about. Someone might care a lot about credibility, I really want to know that this works and might be kind of technically savvy, and so they're a little bit less concerned about the user experience. On the flip side. Maybe someone is more open to a tool that doesn't have as much science behind it, it's really engaging and they like it, and it's really sticky for them.

Schueller: I think again, there's a lot of diversity in terms of what fits for specific people. I mean, I think that, as I mentioned, those VAD, DoD apps are... they have a lot of evidence behind them. We really like them. PTSD Coach. There's a COVID Coach app that kind of focuses on stress and anxiety related to COVID. These apps do lean a little bit more towards veterans. And so there's a little bit more of that flavor in some of their contents sometimes, which might be a barrier for some people. But I think what's great about those apps is that you know that they're based on evidence based practices, they have a solid clinical and development team behind them and that you know that they work really well.

Schueller: I think that for a variety of other apps, there are things that maybe kind of align with what you're kind of looking to get. For example, our group has been looking at mood tracking apps recently. I really like a mood tracking app called Daylio, because it kind of lets me track my moods in a way that's really useful for me, but that might not be useful for everybody.

Schueller: I really do sort of encourage, as I noted before that one of the best ways to kind of find an app that might work for you, go to the One Mind PsyberGuide website, poke around a little bit, maybe find two or three apps in an area, two or three meditation apps that seem like the best picks. Maybe you pick Headspace, Calm and Insight Timer. You look at those different products and you see which one kind of fits best for you. And I think that's really an approach that we sort of advocate in terms of trying to figure out which app a person might end up using to be able to help promote their mental health and wellness.

Mills: I know the health insurance company that I have at work has given us access to at least one mental health app for free, which you would normally pay for. Is that something that's on the increase, where insurance companies are also promoting these apps?

Schueller: Yeah, that's definitely the case. We actually just did a review of that through One Mind PsyberGuide and looked at a couple of different insurance companies and found that there were a lot of different products that they were providing, although nowhere near as vast as the 25,000 I know that are out there. Probably about half a dozen or half a dozen to a dozen products that different insurance companies were providing.

Schueller: My insurance company actually provides one as well. I just got a notification that myStrength, which is a cognitive behavioral therapy app, is available through my insurance. And so that is something worth looking at is that if you have insurance and are covered, there might be a product that your insurance is covering, that there would usually be a payment associated with it, but you might be able to receive it for free. And then additionally, sometimes these are products that are not publicly available on the app store. You may not be able to go download it yourself. They usually go through enterprise contracts to companies or insurance companies or insurance providers, and so you might have some access to some products you might not be able to find out on your own through your insurance company.

Mills: In a recent paper that you published, you wrote about three misconceptions about digital mental health. And one of those misconceptions was that mental health technologies are a new way to deliver psychotherapy. Why is that a misconception? If digital mental health is not a new way to deliver therapy, what is it?

Schueller: I think that technology offers a lot of opportunities to do things in a new and different way. And so I think that if we are merely translating traditional practices from therapy to technology, we are doing a disservice to individuals who might be able to be better reached and better served by new technology.

Schueller: For example, let me make sort of a metaphor here is that a lot of these early technologies out. When they came out, they really had this sort of session frame of thinking that was present and sort of psychotherapy practice, so it's like you log onto the website, you do your 50 minute session, you read something for a couple minutes and you do some exercises and then you have to wait until next week to get your next session. And that's not the way people use technologies. We're a Netflix. We're an on demand world. You want things now and you want things in the moment. If I wanted to learn how to cook an omelet and I signed up for... went to the web and sign up for class, it would be ridiculous to say that, "Okay, well Steven, wait a week and we'll teach you how to crack eggs and wait another week and we'll teach you how to cook them." I want a YouTube video that says, "Show me how to make an omelet right here." And I want to be eating about omelet in 15 minutes.

Schueller: And so I think that technology has the potential to sort of better create these on demand, real world experiences. And I think that those lead to some open questions that we have to sort of further evaluate. I think that we need to think about how clinical practices and behavior change techniques, how the science of what we know what works can be translated to technologies. I think we can't merely make a digital version of psychotherapy, because that's kind of probably the most boring use of what technology's capable of.

Mills: Beyond your work with mental health apps, you're also interested in the intersection of technology and mental health. I think that's what we've been talking about. You recently co-authored a report on teens, social media and mental health that suggested that researchers look at the topic in nuanced ways and not just ask whether social media is good or bad for youth. Can you talk a little bit more about that report and what you found?

Schueller: Yeah, I think that there's a lot of concern, I think reasonably so right now with as much time as everyone's spending in front of these screens, about the impact that screens and social media might be having on people's mental health. And I think sort of more specifically some concerns that the increased rates of social media use are leading to increased rates of things like depression, anxiety and suicide. I want to just say that I think what we've seen looking across the research is that the research has not bearing out that relationship. It does not seem that social media use on average or whole is leading to the increased rates of these things. I think that there's a lot of nuance there that some people are helped by social media, some people are hurt and a lot of people, it's probably not doing much of anything when it comes to their mental health and wellness right now.

Schueller: I think one thing that is really important and that we talk about that report and I think is important in terms of reframing the debate is that screen time as a concept doesn't have a lot of meaningfulness in the current way we use technology. What does an hour of Instagram or Twitter look like? You're reading some content. You're creating some content. You're contributing some content. You're direct messaging with some people.

Schueller: And so I think that what one of the sort of things that we sort of note is that to think about screen time in the same way we thought about it with TV shows, radio shows, things where content is much more chunked and something that you absorb is really different in this new world where you're sort of constantly interacting with these different technologies. And so that we need better ways to sort of understand what is a person actually doing on these platforms when they are interacting with them. And we need to sort of think more about what are the wellness facilitating activities and what are the wellness detrimental activities that might take place on these different technology platforms.

Mills: What do you think the future holds for the use of technology, and not just apps, for the field of mental health? What do you think would be helpful going forward, both for the field of psychology and also for consumers?

Schueller: Yeah. I really think that in 10 to 20 years from now, we're not going to be talking about digital health or digital mental health. We're going to talking about health and mental health and ways that uses technology. I think that technology is going to become deeply integrated into the way that we see services delivered and see people reached. And I think that the current pandemic is really accelerating that. I think we've seen a real uptake in the use of telehealth and virtual visits.

Schueller: And I think to me, the next stage is really thinking about how we create those kind of, as we were talking about a little bit earlier, those new sort of experiences using technology. I think that there's a lot of sort of exciting opportunities for things, like virtual reality, augmented reality, different technologies that really make these things much more pervasive and persuasive and people's lives. And it wouldn't surprise me if in a lot of instances that the first door a lot of people walk through when they think about receiving mental health services or using a tool to sort of promote their wellness or wellbeing is really a digital tool first. I think that these are really potentially scalable cost effective and effective resources. And I think that we'll see a lot more use of these in the sort of continuum of mental health care and wellness support.

Mills: Well, thank you so much for joining us today, Dr. Schueller, this has been really interesting and I hope it's going to be edifying to our listeners.

Schueller: Thanks so much. It was great to be on the program.

Mills: You can find previous episodes of Speaking of Psychology on our website at speakingofpsychology.org or wherever you get your podcasts. If you have comments or ideas for future podcasts, email us at speakingofpsychology@apa.org. That Speaking of Psychology are one word, @apa.org. Speaking of Psychology is produced by Lea Winerman. Our sound editor is Chris Condayan. Thank you for listening. For the American psychological association, I'm Kim Mills.