THIS WEEK, I interviewed Yuval Noah Harari, the author of three best-selling books about the history and future of our species, and Fei-Fei Li, one of the pioneers in the field of artificial intelligence. The event was hosted by the Stanford Center for Ethics and Society, the Stanford Institute for Human-Centered Artificial Intelligence, and the Stanford Humanities Center. A transcript of the event follows, and a video is posted below.

Nicholas Thompson: Thank you, Stanford, for inviting us all here. I want this conversation to have three parts: First, lay out where we are; then talk about some of the choices we have to make now; and last, talk about some advice for all the wonderful people in the hall.

Yuval, the last time we talked, you said many, many brilliant things, but one that stuck out was a line where you said, “We are not just in a technological crisis. We are in a philosophical crisis.” So explain what you meant and explain how it ties to AI. Let’s get going with a note of existential angst.

Yuval Noah Harari: Yeah, so I think what’s happening now is that the philosophical framework of the modern world that was established in the 17th and 18th century, around ideas like human agency and individual free will, are being challenged like never before. Not by philosophical ideas, but by practical technologies. And we see more and more questions, which used to be the bread and butter of the philosophy department being moved to the engineering department. And that’s scary, partly because unlike philosophers who are extremely patient people, they can discuss something for thousands of years without reaching any agreement and they’re fine with that, the engineers won’t wait. And even if the engineers are willing to wait, the investors behind the engineers won’t wait. So it means that we don’t have a lot of time. And in order to encapsulate what the crisis is,maybe I can try and formulate an equation to explain what’s happening. And the equation is: B times C times D equals HH, which means biological knowledge multiplied by computing power, multiplied by data equals the ability to hack humans. And the AI revolution or crisis is not just AI, it’s also biology. It’s biotech. There is a lot of hype now around AI and computers, but that is just half the story. The other half is the biological knowledge coming from brain science and biology. And once you link that to AI, what you get is the ability to hack humans. And maybe I’ll explain what it means, the ability to hack humans: to create an algorithm that understands me better than I understand myself, and can therefore manipulate me, enhance me, or replace me. And this is something that our philosophical baggage and all our belief in, you know, human agency and free will, and the customer is always right, and the voter knows best, it just falls apart once you have this kind of ability.

NT: Once you have this kind of ability, and it’s used to manipulate or replace you, not if it’s used to enhance you?

YNH: Also when it’s used to enhance you, the question is, who decides what is a good enhancement and what is a bad enhancement? So our immediately, our immediate fallback position is to fall back on the traditional humanist ideas, that the customer is always right, the customers will choose the enhancement. Or the voter is always right, the voters will vote, there will be a political decision about the enhancement. Or if it feels good, do it. We’ll just follow our heart, we’ll just listen to ourselves. None of this works when there is a technology to hack humans on a large scale. You can’t trust your feelings, or the voters, or the customers on that. The easiest people to manipulate are the people who believe in free will, because they think they cannot be manipulated. So how do you how do you decide what to enhance if, and this is a very deep ethical and philosophical question—again that philosophers have been debating for thousands of years—what is good? What are the good qualities we need to enhance? So if you can’t trust the customer, if you can’t trust the voter, if you can’t trust your feelings, who do you trust? What do you go by?

NT: All right, Fei-Fei, you have a PhD, you have a CS degree, you’re a professor at Stanford, does B times C times D equals HH? Is Yuval’s theory the right way to look at where we’re headed?

Fei-Fei Li: Wow. What a beginning! Thank you, Yuval. One of the things—I’ve been reading Yuval’s books for the past couple of years and talking to you—and I’m very envious of philosophers now because they can propose questions but they don’t have to answer them. Now as an engineer and scientist, I feel like we have to now solve the crisis. And I’m very thankful that Yuval, among other people, have opened up this really important question for us. When you said the AI crisis, I was sitting there thinking, this is a field I loved and feel passionate about and researched for 20 years, and that was just a scientific curiosity of a young scientist entering PhD in AI. What happened that 20 years later it has become a crisis? And it actually speaks of the evolution of AI that, that got me where I am today and got my colleagues at Stanford where we are today with Human-Centered AI, is that this is a transformative technology. It’s a nascent technology. It’s still a budding science compared to physics, chemistry, biology, but with the power of data, computing, and the kind of diverse impact AI is making, it is, like you said, is touching human lives and business in broad and deep ways. And responding to those kinds of questions and crisis that’s facing humanity, I think one of the proposed solutions, that Stanford is making an effort about is, can we reframe the education, the research and the dialog of AI and technology in general in a human-centered way? We’re not necessarily going to find a solution today, but can we involve the humanists, the philosophers, the historians, the political scientists, the economists, the ethicists, the legal scholars, the neuroscientists, the psychologists, and many more other disciplines into the study and development of AI in the next chapter, in the next phase.

NT: Don’t be so certain we’re not going to get an answer today. I’ve got two of the smartest people in the world glued to their chairs, and I’ve got 72 more minutes. So let’s let’s give it a shot.

FL: He said we have thousands of years!

NT: Let me go a little bit further on Yuval’s opening statement. There are a lot of crises about AI that people talk about, right? They talk about AI becoming conscious and what will that mean. They talk about job displacement; they talk about biases. And Yuval has very clearly laid out what he thinks is the most important one, which is the combination of biology plus computing plus data leading to hacking. Is that specific concern what people who are thinking about AI should be focused on?

FL: Absolutely. So any technology humanity has created starting with fire is a double-edged sword. So it can bring improvements to life, to work, and to society, but it can bring the perils, and AI has the perils. You know, I wake up every day worried about the diversity, inclusion issue in AI. We worry about fairness or the lack of fairness, privacy, the labor market. So absolutely we need to be concerned and because of that, we need to expand the research, and the development of policies and the dialog of AI beyond just the codes and the products into these human rooms, into the societal issues. So I absolutely agree with you on that, that this is the moment to open the dialog, to open the research in those issues.

NT: Okay.

YNH: Even though I will just say that again, part of my fear is the dialog. I don’t fear AI experts talking with philosophers, I’m fine with that. Historians, good. Literary critics, wonderful. I fear the moment you start talking with biologists. That’s my biggest fear. When you and the biologists realize, “Hey, we actually have a common language. And we can do things together.” And that’s when the really scary things, I think…

FL: Can you elaborate on what is scaring you? That we talk to biologists?

YNH: That’s the moment when you can really hack human beings, not by collecting data about our search words or our purchasing habits, or where do we go about town, but you can actually start peering inside, and collect data directly from our hearts and from our brains.

FL: Okay, can I be specific? First of all the birth of AI is AI scientists talking to biologists, specifically neuroscientists, right. The birth of AI is very much inspired by what the brain does. Fast forward to 60 years later, today’s AI is making great improvements in healthcare. There’s a lot of data from our physiology and pathology being collected and using machine learning to help us. But I feel like you’re talking about something else.

YNH: That’s part of it. I mean, if there wasn’t a great promise in the technology, there would also be no danger because nobody would go along that path. I mean, obviously, there are enormously beneficial things that AI can do for us, especially when it is linked with biology. We are about to get the best healthcare in the world, in history, and the cheapest and available for billions of people by their smartphones. And this is why it is almost impossible to resist the temptation. And with all the issues of privacy, if you have a big battle between privacy and health, health is likely to win hands down. So I fully agree with that. And you know, my job as a historian, as a philosopher, as a social critic is to point out the dangers in that. Because, especially in Silicon Valley, people are very much familiar with the advantages, but they don’t like to think so much about the dangers. And the big danger is what happens when you can hack the brain and that can serve not just your healthcare provider, that can serve so many things for a crazy dictator.

NT: Let’s focus on what it means to hack the brain. Right now, in some ways my brain is hacked, right? There’s an allure of this device, it wants me to check it constantly, like my brain has been a little bit hacked. Yours hasn’t because you meditate two hours a day, but mine has and probably most of these people have. But what exactly is the future brain hacking going to be that it isn’t today?

YNH: Much more of the same, but on a much larger scale. I mean, the point when, for example, more and more of your personal decisions in life are being outsourced to an algorithm that is just so much better than you. So you know, you have we have two distinct dystopias that kind of mesh together. We have the dystopia of surveillance capitalism, in which there is no like Big Brother dictator, but more and more of your decisions are being made by an algorithm. And it’s not just decisions about what to eat or where to shop, but decisions like where to work and where to study, and whom to date and whom to marry and whom to vote for. It’s the same logic. And I would be curious to hear if you think that there is anything in humans which is by definition unhackable. That we can’t reach a point when the algorithm can make that decision better than me. So that’s one line of dystopia, which is a bit more familiar in this part of the world. And then you have the full fledged dystopia of a totalitarian regime based on a total surveillance system. Something like the totalitarian regimes that we have seen in the 20th century, but augmented with biometric sensors and the ability to basically track each and every individual 24 hours a day.

And you know, which in the days of Stalin or Hitler was absolutely impossible because they didn’t have the technology, but maybe might be possible in 20 years, 30 years. So, we can choose which dystopia to discuss but they are very close…

NT: Let’s choose the liberal democracy dystopia. Fei-Fei, do you want to answer Yuval’s specific question, which is, Is there something in Dystopia A, liberal democracy dystopia, is there something endemic to humans that cannot be hacked?

FL: So when you asked me that question, just two minutes ago, the first word that came to my mind is Love. Is love hackable?

YNH: Ask Tinder, I don’t know.

FL: Dating!

YNH: That’s a defense…

FL: Dating is not the entirety of love, I hope.

YNH: But the question is, which kind of love are you referring to? if you’re referring to Greek philosophical love or the loving kindness of Buddhism, that’s one question, which I think is much more complicated. If you are referring to the biological, mammalian courtship rituals, then I think yes. I mean, why not? Why is it different from anything else that is happening in the body?

FL: But humans are humans because we’re—there’s some part of us that is beyond the mammalian courtship, right? Is that part hackable?

YNH: So that’s the question. I mean, you know, in most science fiction books and movies, they give your answer. When the extraterrestrial evil robots are about to conquer planet Earth, and nothing can resist them, resistance is futile, at the very last moment, humans win because the robots don’t understand love.

FL: The last moment is one heroic white dude that saves us. But okay so the two dystopias, I do not have answers to the two dystopias. But what I want to keep saying is, this is precisely why this is the moment that we need to seek for solutions. This is precisely why this is the moment that we believe the new chapter of AI needs to be written by cross-pollinating efforts from humanists, social scientists, to business leaders, to civil society, to governments, to come at the same table to have that multilateral and cooperative conversation. I think you really bring out the urgency and the importance and the scale of this potential crisis. But I think, in the face of that, we need to act.

YNH: Yeah, and I agree that we need cooperation that we need much closer cooperation between engineers and philosophers or engineers and historians. And also from a philosophical perspective, I think there is something wonderful about engineers, philosophically—

FL: Thank you!

YNH: — that they really cut the bullshit. I mean, philosophers can talk and talk, you know, in cloudy and flowery metaphors, and then the engineers can really focus the question. Like I just had a discussion the other day with an engineer from Google about this, and he said, “Okay, I know how to maximize people’s time on the website. If somebody comes to me and tells me, ‘Look, your job is to maximize time on this application.’ I know how to do it because I know how to measure it. But if somebody comes along and tells me, ‘Well, you need to maximize human flourishing, or you need to maximize universal love.’ I don’t know what it means.” So the engineers go back to the philosophers and ask them, “What do you actually mean?” Which, you know, a lot of philosophical theories collapse around that, because they can’t really explain that—and we need this kind of collaboration.

FL: Yeah. We need an equation for that.

NT: But Yuval, is Fei-Fei right? If we can’t explain and we can’t code love, can artificial intelligence ever recreate it, or is it something intrinsic to humans that the machines will never emulate?

YNH: I don’t think that machines will feel love. But you don’t necessarily need to feel it, in order to be able to hack it, to monitor it, to predict it, to manipulate it. So machines don’t like to play Candy Crush, but they can still—

NT: So you think this device, in some future where it’s infinitely more powerful than it is right now, it could make me fall in love with somebody in the audience?

YNH: That goes to the question of consciousness and mind, and I don’t think that we have the understanding of what consciousness is to answer the question whether a non-organic consciousness is possible or is not possible, I think we just don’t know. But again, the bar for hacking humans is much lower. The machines don’t need to have consciousness of their own in order to predict our choices and manipulate our choices. If you accept that something like love is in the end and biological process in the body, if you think that AI can provide us with wonderful healthcare, by being able to monitor and predict something like the flu, or something like cancer, what’s the essential difference between flu and love? In the sense of is this biological, and this is something else, which is so separated from the biological reality of the body, that even if we have a machine that is capable of monitoring or predicting flu, it still lacks something essential in order to do the same thing with love.

FL: So I want to make two comments and this is where my engineering, you know, personally speaking, we’re making two very important assumptions in this part of the conversation. One is that AI is so omnipotent, that it’s achieved to a state that it’s beyond predicting anything physical, it’s getting to the consciousness level, it’s getting to even the ultimate love level of
capability. And I do want to make sure that we recognize that we’re very, very, very far from that. This technology is still very nascent. Part of the concern I have about today’s AI is that super-hyping of its capability. So I’m not saying that that’s not a valid question. But I think that part of this conversation is built upon that assumption that this technology has become that powerful and I don’t even know how many decades we are from that. Second related assumption, I feel our conversation is being based on this that we’re talking about the world or state of the world that only that powerful AI exists, or that small group of people who have produced the powerful AI and is intended to hack humans exists. But in fact, our human society is so complex, there’s so many of us, right? I mean humanity in its history, have faced so much technology if we left it in the hands of a bad player alone, without any regulation, multinational collaboration, rules, laws, moral codes, that technology could have, maybe not hacked humans, but destroyed humans or hurt humans in massive ways. It has happened, but by and large, our society in a historical view is moving to a more civilized and controlled state. So I think it’s important to look at that greater society and bring other players and people into this dialog. So we don’t talk like there’s only this omnipotent AI deciding it’s gonna hack everything to the end. And that brings me to your topic that in addition to hacking humans at that level that you’re talking about, there are some very immediate concerns already: diversity, privacy, labor, legal changes, you know, international geopolitics. And I think it’s, it’s critical to to tackle those now.

NT: I love talking to AI researchers, because five years ago, all the AI researchers were saying it’s much more powerful than you think. And now they’re like, it’s not as powerful as you think. Alright, so I’ll just let me ask—

FL: It’s because five years ago, you had no idea what AI is, now you’re extrapolating too much.

NT: I didn’t say it was wrong. I just said it was the thing. I want to go into what you just said. But before we do that, I want to take one question here from the audience, because once we move into the second section we’ll be able to answer it. So the question is for Yuval, How can we avoid the formation of AI powered digital dictatorships? So how do we avoid dystopia number two, let’s enter that. And then let’s go, Fei-Fei, into what we can do right now, not what we can do in the future.

YNH: The key issue is how to regulate the ownership of data. Because we won’t stop research in biology, and we won’t stop researching computer science and AI. So from the three components of biological knowledge, computing power and data, I think data is is the easiest, and it’s also very difficult, but still the easiest kind to regulate, to protect. Let’s place some protections there. And there are efforts now being made. And they are not just political efforts, but you know, also philosophical efforts to really conceptualize, What does it mean to own data or to regulate the ownership of data? Because we have a fairly good understanding of what it means to own land. We had thousands of years of experience with that. We have a very poor understanding of what it what it actually means to own data and how to regulate it. But this is the very important front that we need to focus on in order to prevent the worst dystopian outcomes.

And I agree that AI is not nearly as powerful as some people imagine. But this is why I think we need to place the bar low, to reach a critical threshold. We don’t need the AI to know us perfectly, which will never happen. We just need the AI to know us better than we know ourselves, which is not so difficult because most people don’t know themselves very well and often make huge mistakes in critical decisions. So whether it’s finance or career or love life, to have this shifting authority from humans to algorithm, they can still be terrible. But as long as they are a bit less terrible than us, the authority will shift to them.

NT: In your book, you tell a very illuminating story about your own self and your own coming to terms with who you are and how you could be manipulated. Will you tell that story here about coming to terms with your sexuality and the story you told about Coca-Cola in your book? Because I think that will make it clear what you mean here very well.

YNH: Yes. So I I said, I only realized that I was gay when I was 21. And I look back at the time and I was I don’t know 15, 17 and it should have been so obvious. It’s not like I’m a stranger. I’m with myself 24 hours a day. And I just don’t notice any of like the screaming signs that are saying, “You are gay.” And I don’t know how, but the fact is, I missed it. Now in AI, even a very stupid AI today, will not miss it.

FL: I’m not so sure!

YNH: So imagine, this is not like a science fiction scenario of a century from now, this can happen today that you can write all kinds of algorithms that, you know, they’re not perfect, but they are still better, say, than the average teenager. And what does it mean to live in a world in which you learn about something so important about yourself from an algorithm? What does it mean, what happens if the algorithm doesn’t share the information with you, but it shares the information with advertisers? Or with governments? So if you want to, and I think we should, go down from the cloud, the heights, of you know, the extreme scenarios, to the practicalities of day-to-day life. This is a good example, because this is already happening.

NT: Well, let’s take the elevator down to the more conceptual level. Let’s talk about what we can do today, as we think about the risks of AI, the benefits of AI, and tell us you know, sort of your your punch list of what you think the most important things we should be thinking about with AI are.

FL: Oh boy, there’s so many things we could do today. And I cannot agree more with Yuval, that this is such an important topic. Again, I’m gonna try to speak about all the efforts that have been made at Stanford because I think this is a good representation of what we believed are so many efforts we can do. So in human-centered AI, in which this is the overall theme, we believe that the next chapter of AI should be human-centered, we believe in three major principles. One principle is to invest in the next generation of AI technology that reflects more of the kind of human intelligence we would like. I was just thinking about your comment about as dependence on data and how the policy and governance of data should emerge in order to regulate and govern the AI impact. Well, we should be developing technology that can explain AI, we call it explainable AI, or AI interpretability studies; we should be focusing on technology that has a more nuanced understanding of human intelligence. We should be investing in the development of less data-dependent AI technology, that will take into considerations of intuition, knowledge, creativity and other forms of human intelligence. So that kind of human intelligence inspired AI is one of our principles.

The second principle is to, again, welcome in the kind of multidisciplinary study of AI. Cross-pollinating with economics, with ethics, with law, with philosophy, with history, cognitive science and so on. Because there is so much more we need to understand in terms of a social, human, anthropological, ethical impact. And\ we cannot possibly do this alone as technologists. Some of us shouldn’t even be doing this. It’s the ethicists, philosophers who should participate and work with us on these issues. So that’s the second principle. And within this, we work with policymakers. We convene the kind of dialogs of multilateral stakeholders.

Then the third, last but not least, I think, Nick, you said that at the very beginning of this conversation, that we need to promote the human-enhancing and collaborative and argumentative aspect of this technology. You have a point. Even there, it can become manipulative. But we need to start with that sense of alertness, understanding, but still promote the kind of benevolent application and design of this technology. At least, these are the three principles that Stanford’s Human-centered AI Institute is based on. And I just feel very proud, within the short few months since the birth of this institute, there are more than 200 faculty involved on this campus in this kind of research, dialog, study, education, and that number is still growing.

NT: Of those three principles, let’s start digging into them. So let’s go to number one, explainability, because this is a really interesting debate in artificial intelligence. So there’s some practitioners who say you should have algorithms that can explain what they did and the choices they made. Sounds eminently sensible. But how do you do that? I make all kinds of decisions that I can’t entirely explain. Like, why did I hire this person, not that person? I can tell a story about why I did it. But I don’t know for sure. If we don’t know ourselves well enough to always be able to truthfully and fully explain what we did, how can we expect a computer, using AI, to do that? And if we demand that here in the West, then there are other parts of the world that don’t demand that who may be able to move faster. So why don’t I ask you the first part of that question, and Yuval all the second part of that question. So the first part is, can we actually get explainability if it’s super hard even within ourselves?

FL: Well, it’s pretty hard for me to multiply two digits, but, you know, computers can do that. So the fact that something is hard for humans doesn’t mean we shouldn’t try to get the machines to do it. Especially, you know, after all these algorithms are based on very simple mathematical logic. Granted, we’re dealing with neural networks these days that have millions of nodes and billions of connections. So explainability is actually tough. It’s ongoing research. But I think this is such fertile ground. And it’s so critical when it comes to healthcare decisions, financial decisions, legal decisions. There’s so many scenarios where this technology can be potentially positively useful, but with that kind of explainable capability, so we’ve got to try and I’m pretty confident with a lot of smart minds out there, this is a crackable thing.

On top of that, I think you have a point that if we have technology that can explain the decision-making process of algorithms, it makes it harder for it to manipulate and cheat. Right? It’s a technical solution, not the entirety of the solution, that will contribute to the clarification of what this technology is doing.

YNH: But because, presumably, the AI makes decisions in a radically different way than humans, then even if the AI explains its logic, the fear is it will make absolutely no sense to most humans. Most humans, when they are asked to explain a decision, they tell a story in a narrative form, which may or may not reflect what is actually happening within them. In many cases, it doesn’t reflect, it’s just a made up rationalization and not the real thing. Now an AI could be much different than a human in telling me, like I applied to the bank for loans. And the bank says no. And I asked why not? And the bank says okay, we will ask our AI. And the AI gives this extremely long statistical analysis based not on one or two salient feature of my life, but on 2,517 different data points, which it took into account and gave different weights. And why did you give this this weight? And why did you give… Oh, there is another book about that. And most of the data points to a human would seem completely irrelevant. You applied for a loan on Monday, and not on Wednesday, and the AI discovered that for whatever reason, it’s after the weekend, whatever, people who apply for loans on a Monday are 0.075 percent less likely to repay the loan. So it goes into into the equation. And I get this book of the real explanation. And finally, I get a real explanation. It’s not like sitting with a human banker that just bullshits me.

FL: So are you rooting for AI? Are you saying AI is good in this case?

YNH: In many cases, yes. I mean, I think in many cases, it’s two sides of the coin. I think that in many ways, the AI in this scenario will be an improvement over the human banker. Because for example, you can really know what the decision is based on presumably, right, but it’s based on something that I as a human being just cannot grasp. I just don’t—I know how to deal with simple narrative stories. I didn’t give you a loan because you’re gay. That’s not good. Or because you didn’t repay any of your previous loans. Okay, I can understand that. But my mind doesn’t know what to do with the real explanation that the AI will give, which is just this crazy statistical thing …

FL: So there’s two layers to your comment. One is how do you trust and be able to comprehend AI’s explanation? Second is actually can AI be used to make humans more trustful or be more trustworthy as humans. The first point, I agree with you, if AI gives you 2,000 dimensions of potential features with probability, it’s not understandable, but the entire history of science in human civilization is to be able to communicate the results of science in better and better ways. Right? Like I just had my annual physical and a whole bunch of numbers came to my cell phone. And, well, first of all my doctors, the experts, can help me to explain these numbers. Now even Wikipedia can help me to explain some of these numbers, but the technological improvements of explaining these will improve. It’s our failure as a technologists if we just throw 200 or 2,000 dimensions of probability numbers at you.

YNH: But this is the explanation. And I think that the point you raised is very important. But I see it differently. I think science is getting worse and worse in explaining its theories and findings to the general public, which is the reason for things like doubting climate change, and so forth. And it’s not really even the fault of the scientists, because the science is just getting more and more complicated. And reality is extremely complicated. And the human mind wasn’t adapted to understanding the dynamics of climate change, or the real reasons for refusing to give somebody a loan. But that’s the point when you have an — and let’s put aside the whole question of manipulation and how can I trust. Let’s assume the AI is benign. And let’s assume there are no hidden biases and everything is ok. But still, I can’t understand.

FL: But that’s why people like Nick, the storyteller, has to explain… What I’m saying, You’re right. It’s very complex.

NT: I’m going to lose my job to a computer like next week, but I’m happy to have your confidence with me!

FL: But that’s the job of the society collectively to explain the complex science. I’m not saying we’re doing a great job at all. But I’m saying there is hope if we try.

YNH: But my fear is that we just really can’t do it. Because the human mind is not built for dealing with these kinds of explanations and technologies. And it’s true for, I mean, it’s true for the individual customer who goes to the bank and the bank refused to give them a loan. And it can even be on the level, I mean, how many people today on earth understand the financial system? How many presidents and prime ministers understand the financial system?

NT: In this country, it’s zero.

YNH: So what does it mean to live in a society where the people who are supposed to be running the business… And again, it’s not the fault of a particular politician, it’s just the financial system has become so complicated. And I don’t think that economists are trying on purpose to hide something from the general public. It’s just extremely complicated. You have some of the wisest people in the world, going to the finance industry, and creating these enormously complex models and tools, which objectively you just can’t explain to most people, unless first of all, they study economics and mathematics for 10 years or whatever. So I think this is a real crisis. And this is again, this is part of the philosophical crisis we started with. And the undermining of human agency. That’s part of what’s happening, that we have these extremely intelligent tools that are able to make perhaps better decisions about our healthcare, about our financial system, but we can’t understand what they are doing and why they’re doing it. And this undermines our autonomy and our authority. And we don’t know as a society how to deal with that.

NT: Ideally, Fei-Fei’s institute will help that. But before we leave this topic, I want to move to a very closely related question, which I think is one of the most interesting, which is the question of bias in algorithms, which is something you’ve spoken eloquently about. And let’s start with the financial system. So you can imagine an algorithm used by a bank to determine whether somebody should get a loan. And you can imagine training it on historical data and historical data is racist. And we don’t want that. So let’s figure out how to make sure the data isn’t racist, and that it gives loans to people regardless of race. And we probably all, everybody in this room agrees that that is a good outcome.

But let’s say that analyzing the historical data suggests that women are more likely to repay their loans than men. Do we strip that out? Or do we allow that to stay in? If you allow it to stay in, you get a slightly more efficient financial system? If you strip it out, you have a little more equality before between men and women. How do you make decisions about what biases you want to strip and which ones are okay to keep?

FL: Yeah, that’s an excellent question, Nick. I mean, I’m not going to have the answers personally, but I think you touch on the really important question, which is, first of all, machine learning system bias is a real thing. You know, like you said, it starts with data, it probably starts with the very moment we’re collecting data and the type of data we’re collecting all the way through the whole pipeline, and then all the way to the application. But biases come in very complex ways. At Stanford, we have machine learning scientists studying the technical solutions of bias, like, you know, de-biasing data or normalizing certain decision making. But we also have humanists debating about what is bias, what is fairness, when is bias good, when is bias bad? So I think you just opened up a perfect topic for research and debate and conversation in this in this topic. And I also want to point out that you’ve already used a very closely related example, a machine learning algorithm has a potential to actually expose bias. Right? You know, one of my favorite studies was a paper a couple of years ago analyzing Hollywood movies and using a machine learning face-recognition algorithm, which is a very controversial technology these days, to recognize Hollywood systematically gives more screen time to male actors than female actors. No human being can sit there and count all the frames of faces and whether there is gender bias and this is a perfect example of using machine learning to expose. So in general there’s a rich set of issues we should study and again, bring the humanists, bring the ethicist, bring the legal scholars, bring the gender study experts.

NT: Agreed. Though, standing up for humans, I knew Hollywood was sexist even before that paper. but yes, agreed.

FL: You’re a smart human.

NT: Yuval, on that question of the loans, do you strip out the racist data, you strip out the gender data? What biases you get rid of what biases do you not?

YNH: I don’t think there is a one size fits all. I mean, it’s a question we, again, we need this day-to-day collaboration between engineers and ethicists and psychologists and political scientists

NT: But not biologists, right?

YNH: And increasingly, also biologists! And, you know, it goes back to the question, what should we do? So, we should teach ethics to coders as part of the curriculum, that the people today in the world that most need a background in ethics, are the people in the computer science departments. So it should be an integral part of the curriculum. And also in the big corporations, which are designing these tools, should be embedded within the teams, people with backgrounds in things like ethics, like politics, that they always think in terms of what biases might we inadvertently be building into our system? What could be the cultural or political implications of what we’re building? It shouldn’t be a kind of afterthought that you create this neat technical gadget, it goes into the world, something bad happens, and then you start thinking, “Oh, we didn’t see this one coming. What do we do now?” From the very beginning, it should be clear that this is part of the process.

FL: I do want to give a shout out to Rob Reich, who introduced this whole event. He and my colleagues, Mehran Sahami and a few other Stanford professors have opened this course called Computers, Ethics and Public Policy. This is exactly the kind of class that’s needed. I think this quarter the offering has more than 300 students signed up for that.

NT: Fantastic. I wish that course has existed when I was a student here. Let me ask an excellent question from the audience that ties into this. How do you reconcile the inherent trade-offs between explainability and efficacy and accuracy of algorithms?

FL: Great question. This question seems to be assuming if you can explain that you’re less good or less accurate?

NT: Well, you can imagine that if you require explainability, you lose some level of efficiency, you’re adding a little bit of complexity to the algorithm.

FL: So, okay, first of all, I don’t necessarily believe in that. There’s no mathematical logic to this assumption. Second, let’s assume there is a possibility that an explainable algorithm suffers in efficiency. I think this is a societal decision we have to make. You know, when we put the seatbelt in our car driving, that’s a little bit of an efficiency loss because I have to do the seat belt movement instead of just hopping in and driving. But as a society, we decided we can afford that loss of efficiency because we care more about human safety. So I think AI is the same kind of technology. As we make these kind of decisions going forward in our solutions, in our products, we have to balance human well-being and societal well-being with efficiency.

NT: So Yuval, let me ask you the global consequences of this. This is something that a number of people have asked about in different ways and we’ve touched on but we haven’t hit head on. There are two countries, imagine you have Country A and you have Country B. Country A says all of you AI engineers, you have to make it explainable. You have to take ethics classes, you have to really think about the consequences and what you’re doing. You got to have dinner with biologists, you have to think about love, and you have to like read John Locke, that’s Country A. Country B says, just go build some stuff, right? These two countries at some point are going to come in conflict, and I’m going to guess that Country B’s technology might be ahead of Country A’s. Is that a concern?

YNH: Yeah, that’s always the concern with arms races, which become a race to the bottom in the name of efficiency and domination. I mean, what is extremely problematic or dangerous about the situation now with AI, is that more and more countries are waking up to the realization that this could be the technology of domination in the 21st century. So you’re not talking about just any economic competition between the different textile industries or even between different oil industries, like one country decides to we don’t care about the environment at all, we’ll just go full gas ahead and the other countries are much more environmentally aware. The situation with AI is potentially much worse, because it could be really the technology of domination in the 21st century. And those left behind could be dominated, exploited, conquered by those who forge ahead. So nobody wants to stay behind. And I think the only way to prevent this kind of catastrophic arms race to the bottom is greater global cooperation around AI. Now, this sounds utopian because we are now moving in exactly the opposite direction of more and more rivalry and competition. But this is part of, I think, of our job, like with the nuclear arms race, to make people in different countries realize that this is an arms race, that whoever wins, humanity loses. And it’s the same with AI. If AI becomes an arms race, then this is extremely bad news for all humans. And it’s easy for, say, people in the US to say we are the good guys in this race, you should be cheering for us. But this is becoming more and more difficult in a situation when the motto of the day is America First. How can we trust the USA to be the leader in AI technology, if ultimately it will serve only American interests and American economic and political domination? So I think, most people when they think arms race in AI, they think USA versus China, but there are almost 200 other countries in the world. And most of them are far, far behind. And when they look at what is happening, they are increasingly terrified. And for a very good reason.

NT: The historical example you’ve made is a little unsettling. Because, if I heard your answer correctly, it’s that we need global cooperation. And if we don’t, we’re going to need an arms race. In the actual nuclear arms race, we tried for global cooperation from, I don’t know, roughly 1945 to 1950. And then we gave up and then we said, We’re going full throttle in the United States. And then, Why did the Cold War end the way it did? Who knows but one argument would be that the United States and its relentless buildup of nuclear weapons helped to keep the peace until the Soviet Union collapsed. So if that is the parallel, then what might happen here is we’ll try for global cooperation and 2019, 2020, and 2021 and then we’ll be off in an arms race. A, is that likely and B, if it is, would you say well, then the US needs to really move full throttle on AI because it will be better for the liberal democracies to have artificial intelligence than totalitarian states?

YNH: Well, I’m afraid it is very likely that cooperation will break down and we will find ourselves in an extreme version of an arms race. And in a way it’s worse than the nuclear arms race because with nukes, at least until today, countries developed them, but never use them. AI will be used all the time. It’s not something you have on the shelf for some Doomsday war. It will be used all the time to create potentially total surveillance regimes and extreme totalitarian systems, in one way or the other. And so, from this perspective, I think the danger is far greater. You could say that the nuclear arms race actually saved democracy and the free market and, you know, rock and roll and Woodstock and then the hippies and they all owe a huge debt to nuclear weapons. Because if nuclear weapons weren’t invented, there would have been a conventional arms race and conventional military buildup between the Soviet bloc and the American bloc. And that would have meant total mobilization of society. If the Soviets are having total mobilization, the only way the Americans can compete is to do the same.

Now what actually happened was that you had an extreme totalitarian mobilized society in the communist bloc. But thanks to nuclear weapons, you didn’t have to do it in the United States or in Western Germany, or in France, because we relied on nukes. You don’t need millions of conscripts in the army.

Fuente: www.wired.com