Kim Mills: A few decades ago, the idea of being able to interact directly with a computer using only your thoughts would have sounded like science fiction. But today that technology is here. Commercial companies are testing brain-computer interfaces that allow paralyzed patients to move a computer cursor using their thoughts. Other researchers are exploring prosthetic limbs that patients can control directly with their brains.
Even as these technologies move forward, practical challenges and ethical questions remain. How do brain-computer interfaces work? What uses are researchers exploring for them and what are the biggest challenges they face? What have scientists learned about the brain as they develop these technologies? And in the future, will brain-computer interfaces be limited to medical uses or will we all someday be able to access the internet directly with our thoughts?
Welcome to Speaking of Psychology, the flagship podcast of the American Psychological Association that examines the links between psychological science and everyday life. I’m Kim Mills.
My guest today is Dr. Nicholas Hatsopoulos, a behavioral neuroscientist and professor of organismal biology and anatomy at the University of Chicago. Dr. Halos holds a PhD in cognitive science and a master’s degree in experimental psychology. He studies how our body’s movement is encoded in the brain, and he’s been working on brain-computer interfaces and neural prosthetics for more than 2 decades.
Thank you for joining me today.
Nicholas Hatsopoulos, PhD: Thank you very much. Appreciate it.
Mills: Let’s start with a definition, as we sometimes do on this podcast. What do you mean when you talk about a brain-computer interface or a brain machine interface? What kind of technologies do those things encompass?
Hatsopoulos: So it’s basically any kind of device that interacts with the brain directly. So it could be either feeding information into the brain or receiving information out of the brain. So any, it can span a whole variety of different technologies. I’ve been working on one particular version of it for providing patients with the ability to control devices to move things, but there are a whole collection of different devices that fall under that definition.
Mills: Well, how do these technologies work when a person is controlling a computer cursor with their thoughts? What kind of brain signals are being recorded and how does the computer translate those thoughts into action?
Hatsopoulos: Okay, so what we do is we implant electrodes in a specific part of the brain that’s involved in controlling movement. And we are picking up electrical signals, typically from individual neurons or small groups of neurons, that are firing as a person thinks about moving their limbs. And we extract those electrical signals with these electrodes. We amplify the signals with electronics, and then we send these amplified signals to a computer that then decodes these signals into information that’s useful for the person. So we take this pattern, ultimately these signals are like little pulses because they’re neurons firing action potentials, which are these very brief voltages that last about a millisecond. So it’s almost like a digital computer in some ways because the neuron is either firing, it’s sending the pulse or it’s not. And so we have these pulses, these patterns of pulse activity. We send those pulse patterns of activity through a decoding algorithm that then controls something, whether it’s a cursor, a robotic arm, an autonomous robot, or a variety of other things.
Mills: So what does it feel like for the person using the device? I assume that you talk to people as you’re working with them. And how hard is it for a person to use to learn such a prosthesis?
Hatsopoulos: Well, there really are two kinds of brain machine interface philosophies, actually, if you put it that way. One is what’s called biomimetic, which basically means we’re trying to mimic the way the real biology works. We’re taking the signals that would normally move your arm or your hand and instead move a robotic arm or hand. And that kind of biomimetic approach takes no learning at all. In fact, the subjects learn immediately, and before we work with humans, we work with animals. These animals were able to control devices without any learning, just immediately do it. There’s another philosophy, which is non biomimetic, which is basically taking these signals as if they don’t mean anything. They’re just arbitrary signals that then are sent through an arbitrary decoding algorithm, which then sends commands to the prosthetic limb, let’s say, and the patient or the animal or whatever has to learn through just practice.
In many ways, it’s sort of like what an infant has to do. Infants, they have to, they’re thinking about moving their limbs and they’ve never done this before, and they flail their hands around and they say, oh, if I think this way, my arm goes this way. Let’s say they’re looking for a little toy or something out there. They try different attempts and they flail a little bit with practice. They eventually learn it, and infants learn to sort of reach and grasp objects over the first year of life. So in that sort of similar in that way, which basically these people learn to figure out how to control this prosthetic limb by creating the right patterns of electrical activity in the brain to move the limb the way they want to. And I’ve used both approaches.
Mills: Is there research going on in areas other than motor control and other devices that interact with different parts of the brain besides the motor cortex?
Hatsopoulos: Yes, there is. In the case of movement, there’s another group working on the parietal cortex, which is just behind the motor cortex. And that area also carries movement information, but it seems to be sort of earlier, it reflects sort of the initial intention to move. So when you first intend to make a movement, that area seems to light up earlier than the motor cortex, which then lights up later when you actually execute the movement. So people are looking at devices in that area. It’s not clear which area is better, but both areas work.
Mills: What are the biggest challenges in the field right now in terms of developing neural prosthetics that could be widely available to patients?
Hatsopoulos: I think there are two challenges in my mind. The first is as an engineering challenge, which is developing electrodes, whatever device you’re going to be using for recording electrical signals from the brain or signals from the brain. We need to develop devices that we can put into the brain that are reliable and last for long periods of time, meaning ideally, if you want something to help a person, you want this to last at least a decade without having to go back in and replace it. So there are challenges with that because the brain as a biological organ sees this device as a foreign body and tries to react to it and creates immune responses and tries to impedes the signals that we can record from. So over time, very often our signals get weaker and weaker.
Mills: What can you do to turn off that mechanism in the body? Do you use anti-rejection drugs?
Hatsopoulos: No, no, no. What researchers are trying to do is develop materials that result in less reaction, immune reaction. Materials that are, for example, softer instead of rigid electrodes. They try to build material electrodes that are made of softer substances that almost match the softness of the brain, and that results in perhaps less foreign body reaction. So people are trying to do that right now, but no one has to, my knowledge, has tried to do, tried to eliminate that foreign body reaction or that immune response.
Mills: And if you have something like this implanted in your brain, what did this do? Say if at some point you need to have an MRI, does it have to be removed? Like I was reading, people with cochlear implants can’t have MRIs as long as they have the implant.
Hatsopoulos: With the implant that we’re using, it’s not a good idea to have an MRI. We do MRIs before we implant to localize where we’re going to put these electrodes, but we don’t do it afterwards. So no, it’s not a good idea.
Mills: So I was just wondering about the phantom limb sensations that some people experience after they’ve lost an arm or a leg. Does that phenomenon interfere with using a prosthesis?
Hatsopoulos: We haven’t really experienced that issue. What we do notice is over time, subjects report to us that whatever device they’re controlling, they seem to embody it. It almost feels like a part of their body, and they use words to describe that. For example, in the early days, this is now going back 15, 20 years ago, when they were controlling a cursor on a computer screen initially, they said, well, I’m thinking about moving a mouse with my hand, which then moves the cursor to a certain location over time. They say, well, I’m not thinking about the mouse. I’m not thinking about my hand. I’m just looking at this cursor. I am controlling the cursor directly. I’m not thinking about any intermediate translation from hand, thoughts of my hand, thoughts of the mouse, and then ultimately the cursor. So it’s almost like when you learn a foreign language early on, you translate into your native language and then over time you just speak it and understand it without that internal translation.
Mills: I read about some recent research you’ve done that looked at mapping the movements of the tongue. Why was that something that you were interested in studying and what are the potential applications?
Hatsopoulos: Well, we were interested in the tongue because it’s a soft body. It doesn’t have joints, and so it almost has infinite degrees of freedom. It is just this floppy thing, and yet we’re able to—well, first of all, when we speak, we can control it with exquisite control. We were looking at feeding. And also when you feed and when you eat things, your tongue is moving in various ways. And we were interested to what degree that information about the tongue shape and tongue movement, was that information was encoded in the cortex.
I always thought it was, feeding was something that was done in the brainstem or in lower levels of the brain that we weren’t really conscious of. And it was very basic circuitry that had evolved over many, many, many thousands of millions of years. And the cortex didn’t really care about that because the cortex is usually thought about as the kind of higher area of thought. And so I didn’t think we’d get any information. And it turned out we had lots and lots of information about the tongue shape and movement, and it was remarkable. And we could decode the tongue shape with very good precision with recordings in the cortex. And the application for this, although we are not exploring this at this moment, is for people with difficulties with feeding and swallowing. Dysphagia is a serious problem, especially in the elderly. And so if we could develop a prosthetic that could help particularly with swallowing and feeding properly, that would be a great thing, I think.
Mills: And when you say prosthetic in this case, I mean the tongue is still there, so you’re not replacing the tongue. What is the prosthetic device?
Hatsopoulos: So the prosthetic device would be, for example, when the patient wants to think about swallowing, because swallowing can be done either voluntarily or automatic, but let’s say you want to voluntarily swallow. We pick up a signal from the cortex that says, okay, the person wants to swallow. That’s our trigger. And then we send that trigger to a set of electrodes that we implant in the muscles of the tongue and the hyoid, which is a structure below the tongue that help and assists in swallowing. And we stimulate the muscles electrically to create the proper swallowing behavior. So that’s the way the prosthetic would work, going from the brain to a set of electrodes that stimulate.
Mills: And this is theoretical at this point, is that what I’m hearing you say?
Hatsopoulos: This is theoretical, yes.
Mills: As we’re talking, I mean, it’s clear that this work has important applications. It’s also very interesting as a basic science. What are you learning about how the brain works from this research? What are you learning that we didn’t know?
Hatsopoulos: One thing we’ve learned is how flexible or plastic the brain is, because especially when we’re trying to, we’re using these non-biomimetic decoders, remember I talked about that? That is you take signals from the brain, send them through an arbitrary decoder that it’s not something that’s natural for the subject. The subject has to learn how to activate the right pattern of activity of cells in order to move the device the right way. And what’s remarkable is that both animals and humans can do that. And it doesn’t take that much time, but I mean, it takes some time, but it’s like learning a motor skill, like learning how to ride your bike or play a musical instrument. It does take time, but you can learn how to do this. So you’re taking now a part of the brain that had never been exposed to controlling this device in a completely arbitrary way, and it figures out how to do it. I think that plasticity, that flexibility is remarkable.
And by the way, another thing that other people are looking at as well is you don’t have to focus just on the motor areas of the brain. You can look at other parts of the brain, and people have shown that even in other parts of the brain that you normally wouldn’t think are associated with moving, you can use those again with this sort of non biomimetic approach where the subject or the animal has to learn and the animal learns and figures out how to do it. And so it shows you the flexibility of the brain, which is remarkable.
Mills: There are people out there, probably most famously Elon Musk, who are talking about a future where brain machine interfaces are used for augmenting people’s abilities, not just for helping people who have disabilities. Do you see that happening? I mean, will we all someday be able to access the internet directly with our brains? And is that something that you think we’ll want to do?
Hatsopoulos: I’m not sure we want to do that, but it will probably happen if we solve a problem that still faces us in the field, which is a way to noninvasively implant or record signals from the brain at the resolution we need. So I mean, right now you can use EEG electrodes that are sitting on the scalp and create a brain computer interface. They work not very well. So if you really want a brain-computer interface, you need to access signals at the level of single neurons or small groups of neurons. And there isn’t really a device out there that’s noninvasive at the present moment. And so if it’s not invasive, how many people are really going to volunteer to have brain surgery, open up their skull?
I mean, now there are technologies out there that are several companies out there right now as we speak that are trying to create minimally invasive technologies. So one approach, for example, is you only have to create a small little cut in the skull—very, very small—and slip in a set of electrodes. And these electrodes are sitting on the brain, not in the brain, very high density electrodes. It’s minimal, but it’s still surgery. Another approach is by a company called Syncro out of Australia. They’re trying to implant electrodes through the blood vessels, which then feed up. So they stick that up, I’m not sure exactly where. And somewhere in the body, they stick the electrodes inside the blood vessel, then they feed it up to the brain and then record signals from the brain through the blood vessels, through the walls of the blood vessels. I’m not sure if that’s actually going to work, because you still have, you’re not getting really, really close to the neurons. But those are two approaches that are minimally invasive that might result in larger adoption of this technology. But I don’t know, we don’t have the technology. It just doesn’t exist to actually noninvasively record from the brain at the resolution we need.
Mills: What are you working on now and what are your biggest research priorities?
Hatsopoulos: Our research priority right now is to augment these brain machine interfaces with sensory feedback. So with movement, and this is something I ignored for years, but it’s very important. So when we normally move, we rely, besides vision, which is important, we have to be able to see what we’re doing. But we can do a lot of things without vision. And what’s really important is touch, especially when you’re interacting with objects. You want to touch an object like a cup. You want to make sure you grab it with enough force so it doesn’t slip, but not too much that you break it. So the touch is important.
And another sense called proprioception or kinesthesia, the sense of feeling your limbs in space and how they’re moving and where they are in space. Those two senses are really important for normal movement. And we know this from patients that lack that sense. Those two senses. If you go on YouTube, you can see videos and they’re severely motor disabled, even though their motor system is completely intact. They look like they’ve had a motor injury, but they don’t have those senses. So that’s what we’re trying to do right now.
And currently we’re augmenting these systems with a sense of touch. So what we do is we have now two human subjects at the University of Chicago, and there are others as well at the University of Pittsburgh with whom we work. And these subjects have electrodes both in the motor areas, motor cortex for control, but electrodes also in the somatosensory cortex, which is right next to the motor cortex. And we stimulate electrically instead of record, we send signals into the electrical signals to stimulate those areas near the electrodes. And these subjects report feeling tactile touch sensations on the fingertips. So now we now have what’s called a bi-directional brain computer interface. We’re going both out of the brain and into the brain in both directions. And so with that, you can now imagine a scenario where a subject is controlling a robotic hand. They can reach out, grab an object like a cup, they then feel a cup and manipulate it in a way that they don’t do damage and they don’t break the cup and they feel it. They actually feel what it feels like on their fingertips. So that’s what we’re developing now.
Mills: So we’ve mentioned the sense of touch. We’ve talked about a little bit the hearing because cochlear implants have been around for a while now. You talked about swallowing, but there are several other senses. I’m just wondering, is anybody looking into prosthetic eyes? Will we ever be able to give people back vision?
Hatsopoulos: Yes. So there are several groups looking at artificial vision and yeah, it hasn’t resulted in a commercial product as far as I know, although it’s close. So yeah, there are many groups looking at that. They’re either, there are different approaches to that, either stimulating the retina in the eye or stimulating the cortex, the visual cortex, or something that my colleague, my good friend, long-term friend at Mass General is developing a stimulator that stimulates in between the cortex and the retina called the thalamus and the visual thalamus. And that may have certain advantages. So yes, there’s a lot going on there, but it hasn’t progressed as well or as fast as the cochlear implant, for example. But they’re working on it.
Mills: Well, this is all amazing work. It sounds like magic. And I want to thank you for your efforts. And also thank you for talking with me today.
Hatsopoulos: Thank you for having me. I really appreciate it. It was really fun.
Mills: If you’d like to read more about Dr. Hatsopoulos’s and other researchers’ work on neuroprosthetics, go to the July issue of APA’s magazine Monitor on Psychology at www.apa.org/monitor. You can find previous episodes of Speaking of Psychology on our website at www.speakingofpsychology.org or on Apple, Spotify, YouTube, or wherever you get your podcasts. And if you like what you’ve heard, please leave us a review. If you have comments or ideas for future podcasts, you can email us at speakingofpsychology@apa.org. Speaking of Psychology is produced by Lea Winerman.
Thank you for listening. For the American Psychological Association, I’m Kim Mills.
Nicholas “Nicho” Hatsopoulos, PhD, is a professor in the departments of organismal biology and neuroscience at the University of Chicago. Hatsopoulos earned an MS in psychology in 1991 and PhD in cognitive science in 1992 from Brown University. His research focuses on the neural basis of motor control and learning, and he has been working on research involving neural prosthetic technologies and brain-computer interfaces for more than 2 decades.
