Kim Mills: If you’re the parent of a K-12 student, there’s a good chance your child is using educational technology, or EdTech. Digital tools and platforms are being used to teach everything from spelling to math to science in classrooms around the country. And in the right context, well-designed EdTech can help students learn. But how can you and your kids’ teachers tell the difference between an effective tool that builds real skills and one with flashy bells and whistles that keeps kids entertained, but doesn’t help them master new skills and knowledge?
In September, the American Psychological Association released a report on children’s and adolescents’ learning with educational technology, with 10 research-backed recommendations to help educators, parents, and school leaders decide when and how to use EdTech. Today, we’re going to talk with the report’s lead author about what constitutes effective educational technology. How can you tell the difference between engagement and true learning? How should students’ relationship with EdTech change as they move from elementary to middle to high school? And what special cautions should schools and parents consider when evaluating AI-based tools?
Welcome to Speaking of Psychology, the flagship podcast for the American Psychological Association that examines the links between psychological science and everyday life. I’m Kim Mills.
My guest today is Dr. Nicole Barnes. She is APA’s executive lead psychologist for education and the lead author of the EdTech report. Dr. Barnes earned her PhD in educational psychology from Rutgers University. Before coming to APA, she was a tenured professor at Montclair State University where she studied the psychological foundations of teaching and learning. Earlier in her career, she spent several years as a fifth grade teacher—an experience that she says continues to ground her commitment to educational equity and innovation.
Dr. Barnes, or should I call you Nicole, my colleague? Thank you for joining me.
Nicole Barnes, PhD: Happy to be here, Kim. And yes, please call me Nicole.
Mills: Let me start by asking why did APA decide to put out this report now? What was it about this moment that made you and your colleagues feel it was needed?
Barnes: Great question, Kim. And I just want to start by recognizing those amazing colleagues that were part of this effort. I couldn’t have done it without them. And so all the success that is coming from this report is really because of their collective expertise. So big thank you to all the coauthors.
For me, the inspiration for this report actually dates back to COVID. So at the time, my son Hudson was in kindergarten and like millions of children across the United States, he was learning from home at the desk right next to mine. And one day I looked over and you know what, Kim? I thought he was playing a video game. There were just colors flying everywhere. There was characters that he was dressing. There were all these rewards that were popping up on the screen. And so like any parent during the COVID era, watching their child, I of course, under my breath so that the people who I was Zooming with couldn’t tell, turned to Hudson and said, Stop playing that video game and get back to your math homework.
And do you know what? He did not even skip a beat and he said, Mom, this is my math homework. And wow, did I pause. I can remember like it was yesterday because here I am, a university professor, I had been a teacher myself, and I could not tell whether what I was looking at was a game or a learning experience. And that moment has stayed with me for years. And I have been wanting to do a report on educational technology and the psychological foundations of learning with educational technology since that moment.
Mills: Well, how do you define EdTech in the report? What do you include? What do you exclude?
Barnes: Sure. And I’ll even go a level up here because I think it’s important to first separate educational technology from technologies that children use more broadly—because not every use of technology is educational. So what I mean by that is a child, let’s say watching TikTok on their phone is certainly using a technology, but that is not the kind of technology that we’re talking about in this report. We’re talking about those digital products, those services that are being used or are being promoted as intended to be used to support learning.
And then I’ll actually answer your question that within the EdTech space, there are important differences. Some tools are meant specifically for teaching. So if you think of a tutoring platform or a digital reading program, those tools were designed with instruction in mind and with the outcome of that instruction being learning. Others, we would classify as having a broader purpose, but they’re used to support learning. So think about a video meeting platform or even an app that helps you design your own videos. Those weren’t created for schools, they weren’t created for instruction, but that doesn’t mean that they aren’t used by educators as part of the instructional experience.
So I think a lot of times people will ask me, Does EdTech work? And to me, that seems a little bit like asking, does transportation work? We have bicycles, we have school buses, we have airplanes, and they all work to get people somewhere, but they all serve very different purposes. And we wouldn’t talk about a bicycle and an airplane as doing the same thing. And EdTech is exactly the same way. So important to all you great listeners out there is that what Kim and I are going to be talking about today are those technologies that are intended and marketed for learning.
Mills: Now that application that your son was using during COVID, and you mentioned, and you thought it looked like a game, that was a kind of EdTech, but was it good?
Barnes: Apparently not. And no, I will not name the app that he was using. But I will tell you this, it was intended for learning and it was being assigned as a homework assignment by his teachers. And so that is really what for years just stuck with me as so important to help educate the public and the education sector on, is this differentiation between what a tool is marketed as versus what we actually have the evidence to support its intended outcomes.
Mills: Well, a lot of the debate right now around kids and tech has centered around screen time, but one of the report’s recommendations is to move beyond just screen time and talk instead about what screens are replacing. So why is it important to do this to move beyond screen time and what should we be thinking about instead?
Barnes: I’m so glad that you asked this question because I feel anybody turning on the news right now, you can’t get through a half hour of it without hearing about this screen time debate. And let me just say, I’m a mom. I have two middle school kids. I completely understand why people start with screen time. You can see it easily. It’s easy to count. And I will say that screen time, it might be an appropriate metric for recreational uses of technology like Snapchat or a video game. But when we’re talking about technology for learning, then screen time by itself, it tells us very little about whether a child is learning or whether that experience is helping the child to learn.
So just let’s take an example to play with. If I tell you that a child spends 20 minutes on a screen, how much do you really know about that experience? The child could have been working very deeply through a math problem or like in Hudson’s case, spending 20 minutes dressing a character. So the amount of time is the same, but the experience is not. And so when we’re talking about educational technologies, whether they’re in schools or whether you’re using them at home to help supplement learning, the more useful place to start is what is the child actually doing and what that time is replacing? Those two questions tell us much more than the number of minutes alone.
Now, I won’t say that time doesn’t matter, especially when technology is getting in the way or replacing things like sleep or relationships or other things that we know children need for healthy development. But you cannot, as a responsible parent or educator, treat every minute on a screen as if it has the same value. You need to look at the quality of that experience and the trade-offs that it creates. What looks productive on the screen might not always tell us what the child has learned.
Mills: So can you talk a little more about what parents, educators, and even school districts are looking for when they’re evaluating EdTech to tell if the tool is really going to help children learn?
Barnes: Well, first off, I will say, please do not start with how impressive the tool looks or how many features it has. I would start with the learning, questions like: What is the tool helping my child or our students understand or do that matters? And then I would watch for who is doing the thinking. Is that tool helping the student work through an idea or is it doing so much that the student can move forward without really understanding? I mean, that’s where the real risk is—that it can look like learning has occurred and we really have very little evidence that learning has occurred.
So if it was me, and I promise you, I’m not only talking from executive lead psychologist hat, I’m really thinking about what would be important to me as a mom. I would start to ask myself two questions when looking at my child kind of engaging in EdTech. I would ask, does the feedback that they’re getting through this app prompt my child to think more deeply than simply pointing them towards the right answer? And then if I had some evidence that that was the case, I would ask if there is credible evidence that the tool works for, especially if I’m thinking from an educator’s point of view or from the point of view of a teacher, that that tool works for students similar to those who will actually use it. Because a lot of times the tool is marketed on evidence for one population and that doesn’t mean that it transfers well to another population. And so that claim on a website is more about marketing than it is about evidence.
I would add just one more thing that it’s going to be important for us to start separating activity from impact, especially when we start talking about AI, but this relates to engagement as well. Students can do a lot of clicking. They can look like they finished a lot and they can still learn very little. So we have to make sure these tools are vetted against a clear instructional goal and we have to start listening really closely to teachers and students so that we can figure out whether we have evidence of learning beyond that app. And if the benefit is not there, then schools need to be willing and parents need to be willing to let that tool go.
Mills: How can educators test students to determine if these tools are working as they should? I mean, do they have to design new tests? How will testing need to change in the face of new technologies?
Barnes: This isn’t really as difficult to uncover as we might think. I really, I would urge any parent out there to just look over the shoulder of what your child is doing for no more than 2 minutes. And I think you will be able to tell whether that app is really resulting in learning or not. But the true test is going to be not what happens when the child is using the app, it’s what happens next. So shut down the computer, close the laptop and ask the child, the student, to remember the idea, to explain it in their own words or to use that knowledge in a new situation once those prompts are gone.
Do you remember those keyboards, Kim, that used to—I think they still have them. They used to light up. The keys would light up to tell you what button to press. This was marketed for years as a way to teach folks how to play piano. It’s a little like that. You may play the piano perfectly when those lights are lighting up for you and telling you which buttons to press, but we find out what you actually learned when those lights go away. And that’s the same thing that we’re going to do now. It’s not going to be so much about evaluating what’s happening in the learning experience itself. It’s going to be evaluating what happens after that learning experience and then using that information to kind of then make judgements about how effective that educational experience was.
Mills: Are there certain types of learning or even academic topics that EdTech is particularly well suited for and others that it’s not?
Barnes: In general, EdTech is most useful when it offers a real advantage for the learning task. It should not be the default simply because a digital option exists. So for example, technology can be super helpful when a student needs immediate feedback. Technology can be super helpful when they need practice at the right level or to even give them access to an experience like an AP course or access to a foreign language that would be hard to create in that particular classroom or a school that they’re in. I could even think of in a science class, a simulation that might let a student see a scientific process that’s normally invisible or a practice tool that adjusts as the student improves.
But technology is a much weaker substitute for things like a rich discussion or a hands-on discovery or even encouragement from another person. So I would say that the rule of thumb that I generally use is use the technology when it adds something meaningful to the learning. For learning that depends on complex reasoning, when it depends on relationships, when it depends on working with other people, technology can certainly have a supporting role, but it shouldn’t become the whole experience. And that balance is going to even look different when we talk about to complicate it more than when we talk about different ages of children.
Mills: Does effective ed tech look the same for a kindergartner versus a high schooler? And is there an age that’s just too young for EdTech?
Barnes: Absolutely, yes. Effective EdTech does look different at different ages and we shouldn’t treat it all as the same. It’s like a 6-year-old, a 12-year-old and a 17-year-old are of course going to bring different levels of judgment and independence to the same tool. And we know from the science that as children get older, they are generally better able to manage their attention, they’re better able to question what they see, and they’re better able to use technology more independently. So the tool, it might not need to change completely, but certainly the support around it should. You can think of it a little bit like driving. We wouldn’t hand a 6-year-old and a 16-year-old the same set of car keys and expect the same outcome. I mean, no one would do that. That would just be mind boggling to us. And we can’t do the same with edtech either.
The tool may be the same, but the learner’s judgment and need for support are very different. In the second part of your question, you asked about young children in particular, and I do, I think, want to take a moment to call out some of the risks for young children while also saying that our report focuses on school-aged children. So we did not get into the research on what would be early, early childhood experiences. But I would say what the report hints at is that for young children in particular, we want that digital experience to be brief. We want that experience to be in collaboration or shared with an adult, and it should be connected to something in the real world. Older students, high school students can use technology more independently. They still need to question what it gives them and they need to question how it shows what they know and we can teach them literacy, digital literacy strategies to be able to do that.
But for young children especially, I would say that interaction with caring adults and to the physical world needs to come first. And my advice to parents would say that the younger the child, the harder the tool should have to work to justify its place.
Mills: I want to talk for a minute about artificial intelligence. Explain for us the role of AI within EdTech. And I’m just wondering, because one of the report’s recommendations is to approach generative AI tools with extra caution. Why is that?
Barnes: Definitely. And I will say that although we call out generative AI in one recommendation in particular, an assumption of the report is that certainly as these educational tools continue to develop, we know plenty of instances where AI is either operating in the background or student facing. So we don’t see the other recommendations as being separate from that. The reason why we call out generative AI in particular and add that caution around it is because it can make a final product look better while making it harder to tell how much thinking the student actually did.
Another thing that we don’t talk about too much around generative AI is that generative AI can sound pretty darn confident even when it’s wrong. And that can be especially tricky for children who are still learning how to judge what they see. So I keep coming back to this question, is the learner getting better or is only the assignment getting better? Though the work, it can look stronger, but the student might be less able or less prepared to be able to start, reason or revise without help. If that’s the case, then we have a problem.
I don’t know if you do this, but my mom has just started working with a personal trainer. And we were joking the other day that the point of working with a personal trainer is not to see how much weight the trainer can lift for you. The point is to build what you can do when the trainer steps away. And I think the analogy is, it’s quite similar here. AI, it can have a useful role. It can certainly respond to a student’s thinking after they have made the first attempt. I think that that is super important that the child is making the first attempt. The AIs can offer feedback. They can help the student test an idea, but I want the student to begin the thinking and I want the student to remain responsible for deciding what to accept, what to question and what to revise. I think if AI is acting in that capacity, then it’s making humans stronger, not replacing humans.
Mills: Is there any central clearinghouse, someplace or some organization that’s reviewing these tools and telling educators, this is a good one, this is one you should stay away from? Is that something that APA is contemplating?
Barnes: So there is what’s called an EdTech index. It was an initiative that was funded by philanthropy and supported from a bunch of educational technology associations, not APA, though we do remain a very close partner with a lot of the organizations that were involved. And that is like a clearinghouse where you can go onto the website, you can see a myriad of educational technology tools and you can see the badges that they have earned. And there are different badges for different qualities, like a badge for privacy versus a badge for learner outcomes. So there is a resource available right now that could be very helpful for families and educators who are making these kinds of decisions.
Mills: So you just mentioned privacy. What should families know about how these apps are using their children’s data? How can they determine whether an app is safe from a privacy perspective?
Barnes: Well, I will just tell you from being a mom that I was so surprised by how much information an educational app can collect. I mean, it’s quite frightening actually. It could be not just capturing your child’s personal data, but it’s capturing their work. It’s capturing their thinking and how they move through that tool. And it’s giving them a lot of information about the device and about the learning app. And in some cases, it might even be drawing conclusions about the learner and more broadly to other populations of learners from those patterns. So for me, it’s much more than simply whether a privacy policy exists. It’s whether the amount of information being collected makes sense for the learning task and whether the company is using it for anything beyond those purposes.
So if I was speaking to families or if I was speaking to schools, I would suggest perhaps that they ask a couple, set of questions like: Why does the app need this information? That seems like important to know. Is the information that they’re collecting actually being used to help the learner with that goal of advancing their learning? A second question I might ask is, will it be used for anything besides my children’s learning? And then third, and I would not forget this question, my friends, can we delete it? Do we have control over deleting that information? And if those answers are hard to find or if those answers are hard to understand, I would take that very seriously.
Mills: There are some school districts that are approaching all of this with a lot of caution. Districts including New York City and Los Angeles recently put a pause on using AI-based tools in their classrooms, at least for younger grades. Do you think that’s the right approach or are they just deferring the inevitable?
Barnes: I actually think it’s a very reasonable approach, the pause. And I actually respect quite a lot. I’ve read deeply into New York, less about the others, but what they’re doing to try to approach this in a responsible way. I will say that in the report, we do not call for a blanket ban. We call for age-appropriate policies. We call for paying really, really close attention to what AI is doing to learning. And importantly though, we haven’t really talked about this yet, is we talk a lot about the educator oversight and input that will be needed when the decisions that schools are making could affect a student’s future. I think schools need clear boundaries, but teachers and educators still need to be the ones who are exercising that judgment.
Mills: Tell me more. What about the role of teachers versus technology? I mean, what are teachers adding to learning that technology just cannot replace now and maybe ever?
Barnes: Oh, I do not think that these technologies should replace a teacher. In fact, I think quite the opposite, that the more sophisticated the technologies become, the more important teachers become. And I’ll give you some examples that can hopefully help to paint the picture. So technology gives us information about a student, but a teacher or an educator knows the student, and that’s a very different thing. Teachers see the context around a student’s performance. They notice things like when that child is hesitating, they notice things like when that child is losing confidence or becoming curious or quietly just plain checking out. And I don’t think that a score or a dashboard is going to be, maybe I’ll be wrong and we can play this back at me 20 years from now, but I’ll be in an old lady’s home with —eating ice cream. But I don’t think that that score, that dashboard, is going to fully explain those moments in the way that the teacher can respond to them in real time.
I have yet to meet a technology tool that can noticeably tell whether that child is quietly giving up before that child clicked the wrong answer. But as a teacher, I have noticed that almost every day in the classroom. And I think that even things like a teacher believing in a student and knowing that they’re more capable than what they thought they were, I don’t see technology being able to do that. And great educators do that every single day of their life. So is technology needed? Absolutely. Does it play its role if used well, intentionally and with evidence that it’s doing what it claims it’s doing? Absolutely. And can it extend what a teacher’s work and maybe even impact might be by providing that practice and that feedback or another way into the material? Absolutely. But I don’t see it yet deciding what this particular student needs in this particular moment.
That is a whole child question, not just an academic, where are we on this particular learning objective question? And that’s where I think that teacher judgment and those relationships matter.
Mills: Well, just to wrap up, are there any other recommendations from the report you want to highlight or advice you want to give to our listeners?
Barnes: I guess I would sum up by saying that for us, this wasn’t a report about technology. This is a report about learning, and that is why psychology plays a role here—because we are the discipline that understands how children learn, what motivates children. We understand behaviors, we understand the relationship behaviors and emotion and learning. And so for us, this was really about giving the community, the field, society, the science that helps to help make better decisions. And hopefully you all find this report to be very useful with your children and with your students, and please reach out and let us know if it was. We’d love to hear from you.
Mills: Well, Nicole, I want to thank you for joining me today and thank you for spearheading this important report. I’m sure this is just the beginning of our learning how best to use EdTech going forward.
Barnes: I hope it is. And again, thank you to all of my coauthors. Could not have done this without you.
Mills: For our listeners, you can find a copy of APA’s Expert Report on Children’s and Adolescents’ Learning with Educational Technology on our website at apa.org. You can find previous episodes of Speaking of Psychology on our website at speakingofpsychology.org or on Apple, Spotify, YouTube, or wherever you get your podcasts. And if you like what you've heard, please subscribe and leave a review. If you have comments or ideas for future episodes, you can email us at speakingofpsychology@apa.org Speaking of Psychology is produced by Lea Winerman.
Thank you for listening. For the American Psychological Association, I’m Kim Mills.
Nicole Barnes, PhD, is APA’s executive lead psychologist for education. She earned her PhD in educational psychology from Rutgers University. Before coming to APA, she was a tenured professor at Montclair State University, where she studied the psychological foundations of teaching and learning.
