Most product teams launch new technologies without fully considering their impact on humans, a practice that Arathi Sethumadhavan, PhD, says can lead to serious problems. As senior design research manager on Microsoft’s ethics and society team, Sethumadhavan is always thinking about the societal implications of new technologies, particularly artificial intelligence.
"I work on a team that helps drive ethical and responsible innovation, keeping in mind key ethical principles such as inclusiveness, meaningful human control, privacy and transparency," she says.
Sethumadhavan completed her doctorate in experimental psychology, with a specialization in human factors, at Texas Tech University in 2009. Then, she worked for the medical device company Medtronic, where she helped lead user research for several products, including the world’s smallest pacemaker. Since 2018, she has worked with Microsoft teams that create new artificial intelligence and mixed realityopens in new window technologies—displays that combine virtual and real imagery.
The Monitor spoke to Sethumadhavan about her role.
What do you and your team do at Microsoft?
The ethics and society team is part of Microsoft’s Cloud and Artificial Intelligence division. Our goal is to help scale technology in a responsible fashion by thinking through the implications that the technology can have on people and society.
The kinds of questions that we think about when we design products include What are the harms we foresee with the technology? Would the technology perpetuate unwanted biases or stereotypes? What concerns will customers have about the data that are being collected about them? Are users of the system able to understand how the system makes decisions? I lead user research for this team, so I work with an interdisciplinary group of project managers, designers and product developers to make sure that we are developing technology in a responsible fashion.
What are some of the ethical values you consider?
Microsoft has defined six ethical principles: inclusiveness, fairness, privacy and security, transparency, reliability and accountability. Whenever I start a new project, I analyze the product space and do primary research to answer questions about each of these ethical pillars. Then, I work with the design team to translate the research insights into design recommendations and product prototypes.
For example, take the principle of inclusiveness. For one product designed to enhance team productivity, our goal was to consider how to make the new technology more inclusive for introverted individuals. My job was to dig into what exactly it means to be an introvert. We live in an extremely gregarious society, where generally speaking, extroversion is rewarded. But that does not mean that introverts have lesser capabilities.
So, I did a lot of research to understand: What are the brain differences between introverts and extroverts? What stimulates introverts? What are the unique skills and capabilities that they have that extroverts don’t possess? And how can we capitalize on those unique strengths and abilities through designing better technology? My research revealed that introverts experience more productivity gains when they are given time to reflect. We used this premise to propose design features that were not originally considered by the product team.
In my research, I’m always trying to capture diverse perspectives that traditional product development teams often forget to consider. For example, I work with vulnerable populations—including individuals with speech impairments and people who identify as LGBTQ—to make sure their perspectives are captured during product design.
How did you come to work at Microsoft?
I actually applied for a different job at Microsoft, as the user research lead for the Office Media Group, where I conducted research for applications such as Paint 3D and Stream. But after two months I was reassigned to the ethics and society team, which was formed last August. Lucky for me, I love ambiguity and I love randomness. I joined that team and we started with one big product engagement, then a few small ones came along, and the team continued to grow. Now, I have the opportunity to build my own team and work on a spectrum of interesting projects.
What inspired you to work in ethics and how did you gain the necessary experience for this role?
My PhD is in experimental psychology with a specialization in human factors. During graduate school, I studied how humans and automated systems work together. I examined how air traffic controller performanceopens in new window and situation awareness are affected by varying degrees of automated assistance. I found that when working with high degrees of automation, it took air traffic controllers significantly longer to detect upcoming collisions in their airspace after an automation failure occurred. Highly automated systems tend to leave controllers out of the decision-making loop, making it harder for them to take over manual control.
Ten years later, this problem still exists. Self-driving cars, for instance, take control away from the human—but what happens when the automation fails? It’s important to have a regard for human skills, instead of designing technology that leaves leftover tasks to humans.
After graduation, I got into health care, where I worked in product development for about nine years. When I moved to Microsoft and ultimately joined the ethics and society team, everything seemed to align. I’ve always been passionate about human-centered design versus centering technology products around technological capabilities.
So, now in my job, I get to think about the same issue I studied in graduate school—how do humans and technology interact—but with a much broader lens that considers an individual’s place in society. For instance, one of the spaces that I work in is virtual and augmented reality. Virtual reality allows us to immerse ourselves in fabricated worlds, completely leaving the real world behind. Mixed reality, often called augmented reality, lets us alter or augment the "real world" while carrying on with our day-to-day lives.
Applications of both of these technologies are changing the world, societies and individuals in profound ways. For example, studies have shown that a user’s avatar’s appearance in a virtual world can impact harassment. People can even lose their sense of identity and begin to mimic the avatars they assume in these virtual environments. So it is important to think about not just the benefits but also the risks and ethical concerns of these powerful technologies.
What have you found to be most gratifying about this job? Most difficult?
I’m a curious person, and every project that I touch is very different from the others, so I’m constantly learning about new spaces and thinking through new ethical challenges. It’s also very gratifying to do foundational research that can provide core insights that influence product design and ultimately impact society.
Microsoft is a technology leader, so if we don’t do what’s right, who will? The company’s mission is to empower everyone on the planet to achieve more. To do that well, we need to think about ethics. As psychologists and user researchers, my team is working to understand users’ needs and incorporate their point of view into product design, which I believe is so pivotal.
What’s difficult is also fun: All of these products are phenomenally interesting and also phenomenally new. We can discuss ethical theories and philosophies all day long, but that’s not going to have any tangible impact on product development unless we break it down to what ethics means in this setting. Who would be the most impacted if we create the technology in this way? How do we ensure that it’s fair to everyone? Can people understand how the technology makes decisions? Does the technology exclude certain people? We have to actually think through these problems. There is no formula, so we have to be able to navigate through that ambiguity. That’s the most difficult thing, but it also makes this work extremely interesting.
Do you have any advice for psychologists interested in working in this field?
I do the hiring for my team, and the core skills I look for are a strong background in qualitative and quantitative user research, advanced statistics and human factors methodology—someone who has advanced degrees in experimental psychology, cognitive psychology or human factors.
Psychologists who want to work in product development should also know a variety of different user research techniques, including how to conduct interviews, focus groups, usability testing and large sample-size surveys, as well as how to use statistics to analyze the data collected. But most important, I look for strong written and verbal communicators who can think critically.
Interested in the intersection of psychology and technology? Check out APA’s 2019 Technology, Mind & Society conference, to be held Oct. 3–5, in Washington, D.C. Learn more at https://tms.apa.org.opens in new window
