Harvard Dean David Deming on AI and the Future of Harvard
We host Undergraduate Dean of Harvard College David Deming to talk about his research on social skills, AI, and how they inform his leadership of Harvard
David: I’m making an argument this is like the last human skill.
Andrey: So should Harvard just stop teaching hard skills and go all in on social skills?
David: No.
Andrey: Damn it. I thought I got you. [laughs]
Seth: Nobody’s going to have to communicate anything. The AI will tell each other, and then the AI will just tell me what to do, and no one will ever have to go to a meeting.
David: My guess is it will decrease the quantity of communication but increase the quality of it. AI will not have the meeting that tells you what the company’s strategic direction will be over the next two years, because there’s no right answer to that.
Andrey: If we all do have a lot more free time, then the most important thing that we should be doing is arguing over what the big AI system should be doing for us.
David: We spend a lot more time firing takes at each other and arguing about things, in part because most work is not necessary for human survival. Most of us are entertainers. It would seem crazy to our ancestors that we have podcasts.
Seth: David, I could not disagree more. Welcome to “Justified Posteriors.” Today, we’re joined by David Deming, top labor economist and, since 2025, dean of Harvard College, to talk about social skills in the age of AI and how Harvard is changing to meet the challenge. I’m Seth Benzell, always putting my social skills to the test here at Chapman University in sunny Southern California.
Andrey: I’m Andrey Fradkin, coming to you from San Francisco, California, and we’re very excited to have you on the show, David.
David: Thanks for having me, guys. I’m looking forward to it.
Social skills: coordination costs, the Coasean singularity, and “the last human skill” 1:25
Seth: All right, so I guess the first thing we really want to talk to you about is you have so much really interesting, influential work about how social skills work in the labor market. And so we want to dive into how you think that that’s going to change in the age of AI, how Harvard can be part of that, maybe teach those social skills or select for them. But maybe a good place to start is you have this very influential paper, “The Growing Importance of Social Skills in the Labor Market.” Tell me what that found about change in demand for social skills from the 1980s to 2012.
David: Yeah, sure. So as you mentioned, the paper is called “The Growing Importance of Social Skills in the Labor Market,” and it’s really two things. The paper is an economic model of why social skills would actually be valuable in production and not just like, “Oh, I’m good at cocktail party conversation or I’m good at building social networks.” [laughs] It would actually be productive in the sense that it increases a worker’s marginal product of labor.
Seth: Is the idea that networking would be zero sum?
David: Yeah, potentially. Exactly. So if social skills are networking, it’s just bumping you in line in front of somebody else, where this is actually making you more productive in a competitive economy, right? And that’s not to be taken for granted. And so the model in the paper, the mechanism is, look, production requires specialization. So this is an old idea. It goes back to Becker and Murphy in some sense of-
Seth: It goes back to Smith.
David: Well, okay. Yes, it goes back to Smith originally, but the idea of coordination costs and the idea that basically... So let me just jump back. So Smith says, “Specialization, division of labor is important.” Becker and Murphy, 1992, says, “Look, if specialization makes the economy more productive, why isn’t the economy one big team?” Right? And they notice that if occupations and what people do on the job is actually not only limited by the extents of the market, which was Smith’s argument, it’s limited by coordination costs. And they say, “Look, the economy’s not one big team because it’s actually costly to get together, have a meeting, figure out who’s going to do what.” And the benefits of specialization are increasing and the costs are increasing as it gets more specialized, and there’s some interior condition. And then they explain why, for example, if you live in a small town, you have a family doctor that does everything, but if you live in a bigger town, you’ve got more specialization of doctors, radiologists, and cardiologists, and interventional cardiologists, and whatever, people who put stents in, whatever.
Seth: Passive cardiologists. [laughs]
David: Exactly. So that’s their argument. And so I just take that one step further and apply it to the concept of an individual’s skills. So the idea here is, if you and I are going to work together in a team, maybe we’re going to write a paper together, Seth or Andrey, like we’re going to do something together. And you’ve got some skills, I’ve got some skills, and we’ll arrive at some division of labor. Maybe you’re going to do the data analysis and I’m going to do the writing and whatever. [laughs] And we have some comparative advantage, so if I do something and you do the other thing, we’ll get more out of it. In the same way that two countries trading goods, the idea of Ricardian comparative advantage holds, is like two workers or three workers in our case, trading tasks. So we figure out who’s going to do what. We can do more when we specialize, but there’s some cost. Like maybe it’s a meeting that we have to get together and coordinate who’s going to do what. And so the question is, when do the benefits exceed the costs? And my idea was that social skills are like inverse coordination costs. So the better you are at working with others, the lower cost you pay to realize the benefits of specialization.
Seth: Do you think about that as like a taste? Like some people have a negative friction from interaction? Or is this more like a technology of communication? What do you think about what that penalty is?
David: I think of it as a skill, not a taste. And I think about it like the way I wrote it down in the model was like, if you’re familiar with trade models, there are these what they call iceberg trade costs. So it’s like literally the idea is if you ship a product from one country to the other, some share of it melts along the way. And so-
Seth: The Houthis blow it up.
David: Yeah, exactly. That could happen too. But this explains, these models of iceberg trade costs explain why the US mostly trades with Canada and Mexico, even though China’s a much bigger market. You just got to send things farther away. And so social skills are like inverse trade costs in the sense that we lose less output along the way if we have high social skills, and therefore we can reach finer grain degrees of specialization when we work together. And so I wrote down a model of this, generated a bunch of predictions, including the prediction that social skills will be valuable, but also that they’re more valuable when you have more to trade. So there’s kind of a complementarity between your expertise and your social skills. And so that’s the model side of it. And then I kind of tested it in some microdata using the NLSY, and then also used a bunch of broad labor market data to show that from 1980 to 2012, jobs that kind of require a lot of teamwork had grown faster and paid relatively higher wages over that period of time. And so that’s the paper. So it’s kind of a combination of a model and then a little bit of a theory and then some data that’s tested on it.
Andrey: What are these jobs that require a lot of teamwork?
David: Yeah. Well, I mean, you think about anything in the modern office. I just released a paper about meetings last week. We showed, at least in Norway, the average worker spends an hour a day in the meetings, and that includes workers who don’t have any meetings at all. So, if you’re a construction worker or a retail sales clerk or something, you don’t meet that often. People who work in office jobs tend to meet an average of two hours a day. So, just thinking about meetings aren’t the only way that social skills matter, but the ability to run a good meeting and to learn what you need to learn from that. We all have been in meetings that aren’t run well, and we know how painful that is. Maybe we always think of meetings as a cost, so we don’t appreciate the ones that are run well. But the way I like to think about it and what the model actually implies, which I love, is that higher social skills means you need to spend less time in meetings to get the same output. So it’s like the meeting is the cost you pay to achieve the gains from task trade.
Seth: Right, which would be different than kind of like a taste model where it’s like, “Oh, I’m just loving this meeting. Let’s do more meetings.”
David: Exactly. It’s actually, you can hate the meetings but also get value out of them [laughs] because they are the cost you’re paying. So, you’d like to minimize that cost. So to me, it’s a totally consistent worldview. It’s like meetings are necessary, but I don’t want to spend any more time in them than I have to get done what I need to get done. I think that’s the way most people feel about them.
Andrey: I’m surprised that construction is one where you think it doesn’t require a lot of teamwork, because if one would be-
David: Oh, I just made that up. [laughs]
Andrey: You know? [laughs] Well, I mean, I imagine there’s a lot of coordination, and even if there aren’t formal meetings, people are working side by side, and so they’re meeting all the time, if that makes sense.
David: Yeah, no, I think that’s right. Look, my point was just that the average of an hour a day is an average over all workers, and some people say they don’t really have regular meetings. For people in office jobs or jobs like ours, we probably spend a lot more time than an hour per day on meetings.
Seth: I guess Andrey’s raising a point here, which is like formal meetings versus sort of informal collaboration, right?
David: That’s right. Yeah. And so we asked some questions about that in the survey, but it’s a little hard to get at. You’re basically saying, what is the definition of a meeting? [laughs] And I don’t know. I guess anytime you’re spending, I would say anytime you spend coordinating. So, if you’re a software company, you’re trying to ship some code or ship a product, or if you’re a consulting company, you’re trying to land a client, and you’re spending a bunch of time meeting, figuring out who’s going to do what, who’s going to pitch, who’s going to write the deck. That’s not really production. That’s coordination. And so anything that fits that bill would be a meeting in my book.
Andrey: A natural question here is, I’m hearing that we don’t need meetings anymore because the AIs will meet for us.
Seth: How will this change in the age of AI? Andrey, use the word. Use your term. [laughs] Use the magic words.
Andrey: [laughs] The Coasean singularity.
Seth: We’re going to have the Coasean singularity, David. Nobody’s going to have to communicate anything. The AI will tell each other, and then the AI will just tell me what to do, and no one will ever have to go to a meeting.
David: That might be true, but it also feels like kind of a partial equilibrium conclusion, doesn’t it? I mean-
Seth: Well, coordination costs go to zero. You tell me why that’s not general equilibrium.
David: Well, okay, so here’s a story I would have in mind. Again, I don’t know. Nobody knows the answer because this story’s being written as we speak. But here’s how I would think about how AI changes communication in the firm. So my guess is it will decrease the quantity of communication but increase the quality of it. And you see this in a lot of companies that are very AI forward. They basically have AI hooked up to Slack, which is essentially a within-firm communication technology. It’s like what’s going on in Slack. And when there’s an agent there that you can talk to, you can ask it the questions you would normally ask people, “Where’s the bathroom?” Or like, “Tell me where in the employee manual it talks about my health benefits.” Or, “I heard Seth talk about Project Glasswing in the bathroom. What’s he talking about? Who’s the lead on that project, and when is it shipping?” You know, you can ask factual questions to the AI.
Seth: If I’m talking about Project Glasswing in the bathroom, I’m not talking about a product that’s shipping. [laughs]
David: Okay, fine. [laughs] Fine. Fair enough. Fair enough. But there’s going to be... Look, another way to say it is some of what we communicate about is like there is some answer we’re trying to get to, and everyone knows what it is, and we just have to find the way. But a lot of what firms do and what societies do is communicate over spaces that are contested. So, your AI will not have the meeting that tells you what the company’s direction, strategic direction will be over the next two years, because there’s no right answer to that. It’s like something that has to be determined politically in a sense, like talking with people in the firm. And so that’s the sense in which I don’t see the Coasean singularity as an equilibrium, because the more we lower the cost of routine communication, like just figuring out who’s going to do what, the more space we will open up for communication about things for which there isn’t a right answer.
Seth: Well, I can think of some counterarguments. Andrey, may I go? [laughs]
Andrey: Yeah, go for it.
Seth: All right. Well, thing number one is when people talk about this Coasean singularity idea, they often have in mind political outcomes. They often think about, “Well, I have this long list of preferences about what I would like my hometown to be like. I don’t have the time to research what candidates or which arguments are going to get me there, so I just tell my chatbot agent, ‘Here’s my list of preferences,’ and then my chatbot agent goes to the Agora and then advocates for my angle.” Right? So it’s not clear to me that the fact that it’s political is going to be a barrier to this automation. And then I guess the second idea is just that, I mean, at least in the short term, it seems like we’re getting a lot, in some ways, lower quality communication. It’s so much cheaper to just produce communication that we get more bad versions of it rather than more good versions of it.
David: Which also doesn’t feel like a Coasean singularity. [laughs]
Seth: Well, that is counter. [laughs]
Andrey: To be clear, no one has predicted that it will actually happen. [laughs]
Seth: Andrey, what’s your prior?
Andrey: Well, very low. Less than 10%.
Seth: Very low. Aw, come on. I’m above that. [laughs]
David: I don’t really have a lot of certainty about how this will play out, so I just think it’s not obvious that it will lead to that. And again, like you mentioned, Seth, like let’s say you’ve got your AI doing this and going to the Agora. That all presupposes that you have your argument set and you know what the answer is, and you don’t want to listen to what other people have to say. Like- … part of political engagement means like... And maybe that’s true. Maybe that’s the way society will work. But I actually think you could imagine within a Or some other organization spending more time debating.
Seth: Your AI will pay my AI and will say, “Seth says, I’m willing to accept a 100 dollars an hour of payment to hear your arguments, and there will be a price for communicating to me through my AI.”
Andrey: No, I mean, I do think there’s a vision of this. So I do think that, again, coherent extrapolated volition is a term that gets thrown around in the rationality sphere, but it will know how you would update based on new information. It can do that for you to some extent. You can imagine that. But I do think another version of the future is, if we all do have a lot more free time, then the most important thing that we should be doing is arguing over what the big AI system should be doing for us. And there’s probably going to be just the humanness of we’re going to value that humans are doing the deliberation and not the AIs. So it goes to kind of the relational sector argument.
Seth: Right. This is more like the demand for communication going up. This is the taste argument.
David: That is the taste argument. So I think that argument also holds. I don’t think it’s invalid, but you asked earlier about my paper. Mine was more of a supply side story, whereas I think there’s also a demand side story for relational work, and Alex Imas and others have talked about this, and I think that’s a very valid point of view too. I actually think one way to say it is, let’s look at the trend before AI and extrapolate it outward. We spend a lot more time as a society firing takes at each other and arguing about things, in part because most work is not necessary for human survival. If you think about how many hours do you need to work-
Seth: People would die without podcasts, David. [laughs] Sorry.
David: Well, I mean, it would seem crazy to our ancestors that we have podcasts, because it’d be like these guys are living a life of leisure, or like people make money off of it. That’s sort of my point.
Seth: What was Homer? Homer, “The Iliad?” That’s a long podcast series.
David: [laughs] That’s like one guy. That’s like one guy. Most people were not firing off takes. [laughs] You know what I mean? Or maybe it wasn’t one person, actually. It was a collection of people. But the point is, [laughs] the production of knowledge was much more elite than it is now. And I think it’s in part because what share of the jobs that we do and the work we engage in are actually necessary for the survival of the human race? It’s a vanishingly small share. And so in some ways, when people say, “Oh, AI is going to lead to the end of the work,” to me, I’m like, “It’s already here.” Most of us are entertainers.
Andrey: So, all right, great. So how’s Harvard? [laughs] How does Harvard feel about that take?
Seth: How does Harvard entertain us?
David: That’s a question for you to answer, not for me, how it entertains you.
Seth: [laughs] Well, before we get totally into Harvard, here’s the last question about extrapolating forward your results from a growing importance of social skills, which is you find that there’s an increasing sort of complementarity between having these hard math skills and the social skills. And again, your argument is that you’ve got this more thing to bargain, so you’re going to bargain it more. Do you see the age of AI allowing... Do you see that trend continuing, or do you see people with strong math skills being able to just use the AI if they’re weak on social skills to bring up the bottom, right?
David: So, in the paper, I think the empirical implementation was really about math skills or cognitive skills, but I actually think the lesson’s more general. It’s like you need to be good at something to be worth trading with in these models. But that doesn’t mean smart at math.
Seth: Oh, wait. I thought comparative advantage means you don’t have to be good at anything. [laughs]
David: No, no, no. Comparative advantage means you don’t have to be good at everything. You have to be good at something to have something of value to trade.
Seth: Really?
David: Well, okay, so let’s be precise here. So, there’s a question of whether you can be employable, okay? Which is like, that’s your argument.
Seth: You can be a zero marginal product worker, right?
David: Yeah, but you’re asking about complementarity. So you’re asking about, in other words, the value of social skills is increasing in the value of other skills you bring to the table. So that’s what I mean to say. And so I think that generalizes beyond math skills. You think about your own... I’ll speak for myself. So most of the benefit, most of the good things that have happened to me in my professional life are downstream of becoming an expert in something that people care about. So I spent, I don’t know, 10 years toiling in obscurity, learning about human capital and other things, and you wake up and you’re an expert in some field in economics or something else that people actually want to read about. And then if you’re able to communicate about it generates much greater returns than if you’re not. But if you have nothing interesting to say, having great social skills is not very valuable. So that’s the sense in which complementarity is a much broader concept for these purposes than just math skills. Does that make sense?
Seth: Yes, makes perfect sense, right? So social skills is the way that you trade the other thing you have. It’s not good- … in and of itself, unless we were going to move to a taste-based model in which, “Oh, I just love hanging out with that fun guy.” [laughs]
David: It could also be good in that sense, but that’s not the mechanism that I had in mind in my work. Yeah. Well, we can say more about social skills. So I do think you asked me the question about AI in the future of-
David: And so I think, again, on the supply side, so not relational work, but another way to say it is what is the comparative advantage of people over AI agents? And I argued recently that I think, and I believe this to be true still, that social learning and learning about how people interact together is actually likely to be the comparative advantage for people in a world of super intelligent AI. And I don’t think that’s because the AI is incapable of social reasoning. I think it’s because AI LLMs, their process of acquiring knowledge is just very, very different than how people learn about social networks. Because the social network is impossibly complex. So it’s like there’s millions and millions of possible states of a relatively small network, like who likes whom, who’s superior to whom, who’s... And so there’s just a lot of complexity, and humans don’t map the entire network and then make inferences about it. We basically build causal models in our heads of how people interact, and then we make predictions out of sample and extrapolate like, “Okay, Seth seems cool. He’s likely to get along with his other friend I have,” or whatever it is. And so when people reason, they don’t reason in a way that’s at all consistent with how LLMs do.
Seth: David, I could not disagree more. To me, it seems like having social In the sense of being able to predict how complex social interactions will proceed is similar to the skill of writing a novel about a complex social interaction. Do you think that’s correct, or do you think that’s incorrect, and then that drives why you think LLMs will not have a comparative advantage in that?
David: Let me say what I think LLMs do and what I think people do and why I think they’re different. Okay? And then you can tell me how that fits into the novel framework.
Seth: Please.
David: So I think basically LLMs learn through sophisticated pattern recognition applied to massive amounts of data. So, you need to give the LLM... LLMs are way less sample efficient learners than people are. Okay? But they make up for that in scales. Once an LLM learns, it can replicate easily and it can generalize somewhat, but it takes forever. That’s why you have to feed it massive amounts of training data to get the same basic learning that people get much more naturally. But the learning is much more complete. So people don’t do that. People basically learn by building, let’s say, game theoretic models of each other and reasoning backward. The reason Seth behaves in this way, or the reason he made this face is because this is what he thinks about what I’m saying, and I know something about him, and so I can reason about that. So that’s all, guessing. It’s not building a complete model of you. And then we reason backward from that using incomplete information with pretty sparse data. It’s like we don’t actually know, and we’re conditioned through evolution to make inferences with very little data. And so I think the approach is very different. And so that’s the sense of the argument.
Seth: What do you make of these papers arguing that AIs are getting better and better at persuasion, but they kind of like max out? Do you think this is sort of representative, or do you predict AIs will sort of max out in their ability to persuade? Or do you think persuasion is something that AIs will continue to hill climb on?
David: Well, I think if it’s like written persuasion, like if you write an argument and I have to read it and decide whether I agree with it, they’ll probably get quite good at that.
Seth: And that’s a pretty small part of social skills is what you would say.
David: I would say, yeah. And again, I’m making an argument about comparative advantage. So I’m not saying that, people will always be better than LLMs at this. I’m saying that in a comparative advantage sense, this is relatively hard for at least the modern machine learning paradigm to do, and it’s relatively easy for people to do. So in a comparative advantage sense, I think, again, I’m not making a levels prediction about when AI will be better than people at social skills. Maybe it’ll happen someday. I’m making that argument this is like the last human skill.
AI at Harvard, teaching and selecting for social skills, and the Ivy-Plus lottery ticket 20:39
Andrey: [laughs] That’s a hot take. Well, given it’s the last human skill, let us transition to a question of your institution. So should Harvard just stop teaching hard skills and go all in on social skills?
David: No. [laughs]
Andrey: Damn it. I thought I got you. [laughs] So yeah. So how do you think about what Harvard should be teaching?
David: Yeah. So for exactly this complementarity story. I think that’s the direct argument. So I think it’s you want to build those skills too, but you also want to develop people’s capacities to become an expert in something. And that requires often some technical knowledge. It requires some deep historical reading. It requires, I don’t know, like it depends on the nature of the expertise, but it requires careful study. I think that’s still really important. I just think of it as necessary but not sufficient for success in an AI world. So I think we need to be doing both.
Andrey: I kind of know how Harvard tries to teach non-social skills, but how do you think about Harvard teaching social skills? Is that something that should be more of an overt goal? Does it happen just because it does? Yeah.
David: Yeah, I think the hard thing about that is it’s like social skills, teamwork, leadership, management, it’s like those aren’t really content. Those are pedagogy. So it’s like you can’t say, “Fifth period, it’s social skills time,” and just like start reading out of a textbook. [laughs] It’s more about the way you educate. And so what I think we need to be doing is structuring learning around feedback and reflection much more than we do. So I think one of the reasons why not just Harvard students, but students at a lot of colleges engage more in extracurricular activities than they used to, and more in like work-based learning and project-based learning is because it’s more immersive and because it provides more feedback. Real-world feedback is the best way to learn. I think if all you do is turn in a couple of assignments and you get some cursory comments on your paper, you’re not actually getting enough feedback to learn. And so I’m not saying that it’s happening in all classrooms, but I think the general challenge is like how to bring more of that immersive, interactive learning into an academic setting.
So I don’t know. It’s not going to be one answer for everybody, but I do think we have to reevaluate that. We have to ask, are we giving students enough feedback? Are we giving them enough opportunities to practice what they’ve learned and to build expertise through engagement with each other and with professors?
Andrey: Given that that’s an important part of the coursework, does that argue for changing the admission criteria in some way as well?
David: What do you have in mind?
Andrey: This is a straw man, but you could imagine selecting for people who are the homecoming king and queen of a high school because they have such great social skills and then trying to teach them the harder skills, and then maybe they’ll be really productive, great human beings. I don’t know.
David: Well, I think it’s important to say, at least the way I think about social skills is it’s not really like who wins the popularity contest. So I think of that as conceptually different.
Seth: What do you call that? What’s the term for that? Is that charisma? That’s not a social skill?
David: Yeah, could be charisma. Yeah, I mean, it’s related. These things are not... The correlation between these things is not zero, you know? But when I think about what it means to have good social skills, I don’t think about the person who is most popular or who makes great cocktail party conversation or who... That’s not what I have in mind. I have in mind the ability to work well with others. And that is a skill that requires self-knowledge, but also knowledge of others. So, if the three of us were going to write a paper together, we would arrive at a division of labor that is different than if I were to write a paper with one of my students or one of my advisors or what. So, there’s no one answer to it. It’s about adaptation to others. And flexibility. If I thought there were a really good way to measure not even just the skill, but like whether people are receptive to it and whether it can be built in some people more than others, I would absolutely want to try to find a way to select on that. I don’t think we’re there yet. So, I wouldn’t want to noise trade on things that we feel like are social skills but really aren’t.
Seth: Do you think Harvard should only admit on SAT scores?
David: No.
Seth: And the reason for that has something to do with social skills, right?
David: No. No, I don’t. And I think it’s broader than that. I think what not just Harvard, but any university is trying to do is admit a class of people who are the future leaders and contributors to society in a variety of ways. So I don’t think Harvard’s model should be or is like, we admit only people who score high on certain tests because that’s the only thing that matters. I think it’s more like we want people who make contributions to society, and those contributions take many forms. I think academic excellence is important, so I think it should be a very important part of our admissions, but I don’t think it should be the only part. And part of that too is honestly, Harvard is fortunate to be in the position of basically being able to admit, we could admit a class full of people who have close to perfect academics. And so you actually, in some cases, don’t need to choose. You can get people who are great academically but also have other skills, and many of our students fall in that category.
Andrey: I guess, one of the comments that you hear is that part of the problem is that SAT is just not hard enough. It’s too top censored, right? You could imagine a test that’s much more comprehensive, like they have in India, where you could really spread out the top talent. You’re curious what you think about that proposal.
David: Well, that’s another reason to not, I mean, Seth asked, should you admit just on SAT scores? That’s another reason not to do that because it’s top-coded. And that’s why we don’t do that, or one of the reasons why we don’t. I think there’s a lot of other ways to detect academic excellence beyond that. It’s not just tests. It’s also like what were your courses, what other enrichment you know, like in the case of math, like there’s well-known indicators of extreme math potential. And we have faculty read applications who are experts in fields, and so that’s part of the admissions process too, is like people who say, “Oh, this person’s going to win a Fields Medal, we think.” And like, that matters too, and that’s way above the scale of the SAT. And so, that’s a part of admissions at not just at Harvard, but many other schools, and I think it should be.
Seth: You made a really interesting comment, which is this idea of thinking about correlations between different social skills and maybe hard academic skills. And one thing I noticed in that your growing importance to social skills paper, I don’t have the number in front of me, but I believe you found approximately a 25% correlation between AFQT score, which is kind of a general IQ test, and social skills. If we’re moving to a world where social skills are more important, do you think that that kind of reorders the winners and losers in society? Or are those correlated enough that the people who would have been the winners in one society are going to be the winners in the other society?
David: Yeah, that’s a good question. It may reorder them. Not necessarily on raw skills, but I think the kinds of jobs. So I do think the winners and losers in the economy, not just in schools necessarily, will tilt more towards people who, I wouldn’t say just like have good social skills, but people who know how to sell and persuade and negotiate and navigate conflict. These are, I think, increasingly important skills that are kind of subsets of what you might call social skills. And I think they can be developed. So I think people are like, I think to me, like probably social skills are like any other thing, which is like some people are born with different endowments, but you can work on it. For me, like having children, especially daughters, has definitely built my social skills, no question about it. Having a job like this where I have to talk to people all day has definitely built my social skills. So I think you can work on those things, and I think you’ll see more people spend more time working on these things because it’ll be rewarded.
Seth: Well, tell me what that looks like in Harvard. Tell me what that looks like outside Harvard. Is this people going to Toastmasters? What is that?
David: Well, I would guess, this is a guess, I don’t have data on it. If you were to scan the syllabi of courses in any college and measure the change in assignments over the past 10 years, I would bet anything there’s more oral presentations, there’s more project-based assignments, and there’s fewer blue book exams or pen and paper tests, and probably even fewer papers. And so that I think is a recognition of this. And I also think a lot of the extracurricular activities that students engage in, certainly at Harvard, are like, they’re very interactive. So there’s a consulting club that basically has real clients that they make pitches to and write things for and they try to generate a book of business. And so, that’s something that they’re not really getting in the classroom, or at least not as much. So I think you see, when you start looking for it, you see all kinds of signs of the demand for this and people finding ways to satisfy it. So I think that’ll probably continue.
Andrey: All right. So Harvard has over 40 varsity sports. A large share of the student body in the undergraduate plays these sports. But at the same time, America’s reliance on college athletics or the prevalence of it has been criticized. What is the optimal amount of college athletics that you think colleges should have?
David: I don’t know. I think athletics are an important part of what we do. I can’t speak to other universities, but here at Harvard, I’m pretty proud of the fact that all of our athletes live in the same dorms with students and have the same college experiences. So I think we’re really about building student athletes. Our athletes are pretty high achievers. Not all of them, but most of them are. And they meet the Harvard admission standards. And so I think of it more as they’re great athletes and great students. And I do think athletics builds social skills, discipline, a sense of sacrifice. There’s good things that come out of athletics. Whether that means we should have more or less on the margin, I’m not really sure. I don’t know. I see both sides of it. I see schools that have less athletics than Harvard but are also academically competitive, and that means they can, I don’t know, they have more degrees of freedom. But I also think our athletes do actually do exceptionally well on the job market, and some of our teams are pretty good, and that brings a lot of good things to the university as well, so.
Andrey: I’ll just editorialize a little bit because I assume you’re not at liberty to say. I Plenty of sports, not all sports, are a way for colleges to admit the children of very wealthy parents. That might even be optimal for a university, even though it is not fair. Okay. Editorialization complete. [laughs]
David: Thank you for registering your opinion.
Andrey: Now, you did not go to Harvard for your undergrad.
David: I did not. The Ohio State University.
Andrey: First of all, just a biographical question. Why did you end up going to OSU?
David: Well, I went to Ohio State. I moved to Cleveland with my family when I was 15 from Nashville, Tennessee. So, I attended Shaker Heights High School for three years, which is a public school on the east side of Cleveland. Actually has a storied history. It’s a racially integrated school, and through some deliberate gerrymandering of school districts in a way to preserve racial balance in the school that’s very interesting, it’s been written about. So, I went to Shaker, and then I went to Ohio State on an academic scholarship. They were running a program at that time. Basically, if you were a National Merit finalist, you could go there for free and they would pay you, I think, 3,000 dollars a year or something, which felt like a lot at the time. And so, I went there to save my family money, and also I went to the honors college there and had a great experience. And I got into some other schools. I think I got into Vanderbilt and Emory. I got some money to go to those schools, some financial aid, but it felt important to me to save my parents some dough, and I also thought I could get a good education, and I did.
Andrey: Awesome. So, our question to you is, how do you think about the Harvard College premium for undergrad? I know you’ve done some work on this. Maybe you can recap some of that work, and then we can talk about why do you think there is a premium? Where does it come from? Yeah.
David: Sure. Yeah, so I think you’re referring, Andrey, to a paper that I wrote with Raj Chetty and John Friedman about... The paper had two parts. Really was one is like who gets in and why, so the determinants of admissions at elite colleges. And the second one was like, what are the causal impacts of attending one of these schools? And in that second part, which is what you’re referring to, we found is that basically there was a small impact on mean earnings. It was not tiny, but it was not earth-shattering. So if you go to one of these schools... And just to say, our identification strategy, which was not perfect but pretty good, was like we looked at people who were on the waitlist. So we had data from several Ivy Plus colleges that declined to be identified. We first sort of used the waitlist to verify that just because you’re on the waitlist at one doesn’t mean you’re on another. We kind of looked at people who were kind of right on the margin of admissions, controlled for a lot of stuff, and showed what if you barely got off the waitlist and compared waitlist admits to waitlist rejects.
So, what we found was that being admitted to a school off the waitlist, to an Ivy Plus school, relative to a public flagship like an Ohio State, some place that I went, led to a small increase in mean earnings, but a very large increase in the probability of being in the top 1% in your cohort. So, impacts on the top tail that were pretty large. It’s been a while, but I think it was like a 50% increase in the probability of being in the top tail of earnings, top 1%. And there’s kind of a fractal quality to it, so we don’t have enough statistical power to go all the way up, but when you just look at the distribution, it looks like the increases are all driven at the top. And if anything, there was a small increase in the probability of having zero earnings. So, that’s people who are in grad school or doing something else. So it was more of a fanning out, but an overall mean increase, but that’s driven by the top tail.
So, that was what we found in the paper, and we think it’s very much connected to the advantage that people have in landing at what we would call an elite or a prestigious firm. And that has a loose definition, but you should think about it as like, if you go into finance and you go to Ohio State, you get a job at State Street Bank or KeyBank or something. And if you go into finance at Harvard, you go to Goldman Sachs. So they’re these big firms with national reputations where it’s very hard to get a job and they only recruit at a few schools. But it’s far from a guarantee. So I like to think about it like, if you get admitted to an Ivy Plus school, you get a lottery ticket and you have maybe a one in 10 chance of cashing it. But if you went to Ohio State, you don’t get any lottery tickets. So it’s not a guarantee, it’s just you get more tickets.
And if you want to win the tournament in a high-status profession, not just finance, but technology or consulting or journalism, if you want to get a job at the New York Times rather than the Columbus Dispatch, it’s also a gateway to that. So, our story is that basically these colleges allow you to buy a lottery ticket or two in a tournament that few people win, but the pipeline, the opportunity set for winning that tournament greatly increases if you go to one of these schools.
Andrey: And is that just because of access or is that because of human capital formation or... Yeah, do you want to speculate on the mechanism there?
David: I could speculate, but just to be clear, our identification strategy doesn’t allow us to answer that question really. My own sense is it very much depends on how broadly you’re willing to define human capital. Okay? So, there’s a narrow sense in which human capital is just years of schooling, [laughs] or quality-adjusted years of schooling. That I think is probably not the mechanism. But I think it could be that you get an early start on learning a very specialized set of skills. And I’ll just give a concrete example. Again, I mentioned the consulting club. So these are clubs that where you actually learn how to be a consultant [laughs] while you’re in school. And so that’s human capital. It’s quite narrow human capital. But then also, so it’s kind of endogenous. The fact that these firms, like McKinsey hires a ton of people from Harvard, is endogenous to this. So it’s like you as a student, you know what they want, they know what you want, they get to know you because they think you’re the most talented. So it’s like there’s some soup of human capital development and networking and signaling that all goes together that I think is pretty hard to unpack. So that’s my own sense. What’s your sense?
Andrey: My sense? [laughs]
David: Yeah, I mean, well, the interpretation, you have just as much a claim on that as I do. I wrote the paper and I did the analysis, but that doesn’t mean I know what the mechanism is.
Andrey: I think in some places like grads, to the extent that Harvard undergrads end up accomplishing a lot in academic fields, they are able to get access to the top faculty earlier on. They’re able to essentially get their research accelerated. I think for things like banking and consulting, I think in addition to the human capital formation is peer effects. Peer effects on what is worth desiring as a career outcome. If your friends are also very interested in banking, you’re going to be talking to them about the banking, and you’re going to do much better in the banking interviews, and you might even be a better banker as a result. So I think there’s a lot of that going on. In my experience as an econ major at a university that produces a lot of investment bankers, I don’t think actual knowledge of economics was the key determinant of who got those jobs, so yeah. [laughs]
David: Well, so that’s why I say I think it depends on what you mean by human capital. So I think I agree with your analysis. The way I would say it in my own experience, too, is having gone to a school like Ohio State, and then I came to Harvard to do my PhD, the big impact of Harvard on me, I mean, which led to human capital formation, was it raised my expectations for myself. I had a classroom of people who were like, oh, I was like, “These people are serious. I better get my act together.” You know? And it made me learn more. So is that human capital? Because I built more human capital causally because of that, but it was driven by a peer comparison. So I think it’s just hard to disentangle. I think the way we talk is human capital signaling networks. It’s just sort of hard to put any of it into that box so neatly. That’s my sense.
Seth: I’m going to try to have an AI take about this. I guess the AI take is, I think the idea of forming personal connections and being in a personal network with a bunch of people who are in a community will become relatively more important as a mechanism for getting these prestigious firm outcomes, where it’s really, in some ways, more important to have a community of people who trust and have a common background with each other than it is necessarily about skills. Because if we’re able to outsource more and more of our skills to the AI, it’ll be about tribe formation, and I think that’s a big part of what goes on at some of these elite schools.
David: Yep. I think it’s true.
Andrey: Yeah. So let’s talk a little bit about AI use policies at Harvard. Do you want to kind of lay out, first of all, your position on it, and then we can ask you some questions?
David: So I wrote a convocation speech that I delivered to students on the day before the first day of classes. I talked about the three eras of undergraduate education. My argument was that we’re entering a place because of AI, but not only AI, where we need to embrace a more communal aspect, and I talked a little bit about how that relates to AI policies. I sent an email to students the first day of class outlining my goals for the year at the college, which is something I did last year too, to say, “What are we trying to accomplish?” And one of them was trying to work with faculty to develop a set of advisory guidelines about AI use in classes. And I think they have to be guidelines and not hard policies, because the way you use AI in an intro to computer science class and a 10-person humanities seminar is just totally different, so I don’t think there’s a one-size-fits-all approach. But I do think we need to give more guidance to faculty and to students about this, because the state of nature here is that everyone’s using AI all the time for everything.
And it’s actually quite easy for me as a 47-year-old man who built my skills in the pre-AI world to put AI down and write for myself. It’s actually not that difficult. But for students who are basically growing up in this era, so our college freshmen now have been through most of high school with this tool available, I think we can’t take for granted that it’s as easy for them. And I think what’s happening is, in a lot of classrooms, faculty are saying, “You’re not allowed to use AI,” or, “You’re allowed to use it for brainstorming, but not for writing,” as a loose guideline, but not in an enforceable way. And then you come and turn in a term paper, and students are just put in a difficult position of doing the right thing, like behaving with integrity, but feeling bad about it because they think their classmates are not behaving with integrity, and maybe their classmates are getting better grades and using AI on the sly, or violating the course policy.
So you either do the wrong thing, or you do the right thing and feel bad about it. And so I think you have to be realistic about what’s happening out there right now. And many people will do the right thing, so it’s not that I think students are going to cheat all the time, but enough of them will, or at least they’ll feel the temptation to, that we have to design our assignments and courses to be AI aware, AI accepting, if we’re not willing to take the steps necessary to enforce it. So I think you could have, on one level, you can have a low-tech equilibrium where everything’s blue books and it’s controlled conditions, and that’s a totally fine way to do a lot of teaching and learning. And I think on the margin, we should be moving more courses toward that. Like in Ec 10, Jason Furman and David Laibson are teaching introductory econ here at Harvard, and they reduced the weight on take-home problem sets this year and increased the number of midterms.
So that’s an AI adaptive thing in one direction. In the other direction, if you want to assign a paper and give it to people to take home, you just have to understand that AI is going to be there for them. And I think there’s a lot of ways to do this. I don’t have all the answers, Seth, or Andrey, but to think about it conceptually, I think as a professor, you need to get more into the process of writing with people if you want to be able to guarantee that people are doing the thinking on their own. So one thing I’m going to do in my class, which I’m teaching in the spring, is I’m going to ask students to write a first draft under controlled conditions that we know is AI-proof, they’ve done the thinking for them, and then they can use AI to help revise and draft under supervision. So it’s like, turn in a first draft, turn in a second draft, give me an outline first, tell me what you’re thinking.
Oral exams, spot checks on, “Oh, you developed that argument. Tell me more about it.” So it’s really just getting into the guts of the thing, which I think will actually be better for student learning anyway. So that’s an example of how you could encourage AI or you could accept AI without giving over the thinking to the tool, which I’m very much against. But more than anything, I just want us to start talking about this more than we currently are and not just pretending like it’s not happening.
David: Because I talk to students all the time who are quite anguished about this. They want help.
Andrey: Two things you mentioned, like one, controlled setting. Do you think there’s a possibility of creating a library space or somewhere where people can go and access a computer guaranteed, no internet, no AI? Would that be useful for students? You kind of clock in, you drop off your phone, you go sit there for a few hours, and you just write?
David: Yep. Some students do this on their own. I basically bought a bunch of these phone pouches, these Yondr phone pouches, that we’re going to have available on a voluntary basis for students who want to enter a space. And I don’t know if you know how they work, but you basically have a little box and you stick the thing on it, and you carry your phone with you, so it’s not me taking it from you, or anybody taking it from you. It’s just like a voluntary commitment device. So we’ll see, maybe that people want that and maybe they don’t. I’m just trying to experiment with different ways to just make it a little easier for students to do the thing that they want to do. And I was telling some students last night, I always feel torn between two principles. One is, there’s certain things that I would like for them to do, [laughs] but also they’re adults and they make their own decisions, and why should they listen to me? So I’m trying to make it a little easier for them to do the thing that they want to do while also not being too paternalistic.
Andrey: What about oral exams? So, I can imagine a world... One of the biggest kind of source of anguish for faculty is the fact that we have to both teach and evaluate, right? And that puts us in a very unusual, strange position to be cops. And cops in different ways. One, catching cheating, which is just not a very pleasant experience for everyone involved, but also students arguing about grades and so on. Let me imagine sitting a student down with a computer, and the computer is talking to the student in an oral exam fashion and is evaluating them. And that’s something that I can imagine can be done in most undergraduate classes for sure without faculty involvement. Have you considered this sort of mechanism? Do you think we’ll see it?
David: I’ve talked to faculty actually who are trying things like this. And if somebody handed me a tool like that that worked really well, I’d be pretty open to certainly to using it in my own class. I think basically the oral exam is a great technology for assessing mastery. That’s why we do oral defenses in PhD. And I don’t know how you guys do them or have done them, but I tend to just like, students will submit a research statement or paper and I just keep asking questions until they don’t know the answer, and then we talk about that. And that’s a very effective teaching and learning tool. The problem is it doesn’t scale very well, to your point. And so if you could use technology to scale it, I think that’d be great. Like a adaptive, personalized oral examiner would be a great way to learn. So I’d be pretty open to it. I haven’t seen anything that’s shovel-ready, but that doesn’t mean it doesn’t exist. Maybe somebody will watch this and send something to me and I’d be open to using it. Or maybe you have it. I don’t know.
Andrey: I don’t. But yeah, I think that’d be pretty great.
David: That’s disappointing.
Andrey: Yeah. [laughs] I think there’s a professor at NYU who I saw wrote a blog post about his-
David: Really?
Andrey: … design of such a technology.
David: I think it’s a cool idea. And it would be mildly surprising to me if in a few years people aren’t using something like this more. It just feels like an obvious win. Education way back was tutorial style. Exactly. So, why not go back to that?
Andrey: Well, I think one of the reasons we stopped doing it is it was kind of too subjective. There was too many ways for there to be bias. I have personal history of this. My parents were in the Soviet Union and are Jewish, and they never got the same oral exam that the normal students got.
David: That’s a good point. Yeah.
Andrey: Yeah. But I think with an AI, you’re a lot less worried about these sorts of things.
David: So, as with anything, technology that solves some problems creates other ones. But it’s an exciting area to push on, I think.
Grade inflation 46:05
Andrey: Now we were talking about assessment. I would love to ask you a little bit about, there’s been this huge debate about grade inflation in American universities and at Harvard, how to deal with that. How do you think about this problem?
David: So at Harvard last year, the Faculty of Arts and Sciences voted, I think by a 70 to 30 margin, to cap the share of As, not As and A minuses, but just straight A grades in Harvard College classes, to 20% plus four. So what that means is, like in a five-person class, you can give 100% As, in a 10-person class, it basically asymptotes to 20% as the class gets larger. And also, less commented on, but just as important, voted on something called the average percentile rank, which is basically in order to calculate honors for students to break ties, you basically rank students internally in the class and then use that to calculate honors, not just whether you got an A or an A-minus. So it gives more gradation to people’s performance in a way that allows you to figure out who deserves some magna whatever. So those are the two things they passed. I should say, I was the dean when it happened, but this is a multi-year effort.
I supported it. I was actually for it, but it wasn’t my initiative. It was something that the faculty worked on through a committee over the course of the year, and I kind of just shepherded it through. But I do think it was good. And when I say good, I mean better than nothing. So my view on this, and maybe this is just like, maybe you guys will be in a role like this one day, maybe not, but what I would say is you end up living in a second best thinking on the margin world a lot more than you’d like to. So I answer a lot of questions from students and faculty and other people who are sort of like, “Why didn’t you do X?” And I’m sort of like, “Well, X is might be what we wanted, but Y is what we could pass and it’s better than Z.” So like this, when you talk to people about what is their ideal grading system, people have lots of ideas.
I think some of them were good, some of them weren’t very good, in my opinion, but everybody’s got their own ideas. And then you have to triangulate that against, what do you think students will tolerate and what will the faculty pass? And so what we did was turn this over to a committee of faculty who spent a bunch of time talking to students and thinking about it, and then they developed this proposal, and it was like yes or no. And so I kind of said to people, “Look, there are a lot of things that I might do differently about it, to be honest, but I think this is actually going to make things better than not doing it.” And that’s because we basically were... My view is if you look at the trend, we were basically increasing the share of As by three to five percentage points a year. And once that gets up to 100%, which it would in a decade, we’re functionally a pass-fail institution.
And if everyone’s getting As have no meaning. And so that’s what I think we needed to prevent, and this is a way of doing that, even if it’s not the best way. A question that people asked a lot was, shouldn’t an A be a standard of mastery? You know? So it’s like, well, if everyone learns the material, everyone should get an A. And so actually there’s an answer to this in the handbook, which is that the grades of A range are reserved for assessment of mastery. So, anybody can get an A or an A minus, but the grade of A is rewarded for performance of extraordinary distinction. That’s the language in the handbook. And so that means if 60% of people are extraordinarily distinguished, then it doesn’t have any meaning. And so the faculty said, “Look, A-minus is for mastery. Anybody can get it if they meet the standard. But A is for both mastery and distinction.” So there’s at least a logic to it.
Andrey: So let’s think through the equilibrium here. It’s going to affect a few margins. Student effort in a class, right? Just presumably the amount of learning. Student choice of classes, and then employer interpretation of the signal of the grade. So have you started monitoring some of these? What’s going on after the policy has changed?
David: So on the employer side, we monitored it ahead of time. So one thing that Amanda Claybaugh and I did, and she’s the dean for undergraduate education, she really shepherded. She was in that role before I was, and so she really is responsible for a lot of this. She talked to a bunch of employers to get feedback on this, and I would say they were 100% in favor of it, actually. And what they said was, “Your grades don’t mean anything to us, and so we have to look for other signals of distinction,” basically. And we heard that from graduate school admissions directors, too. And they don’t just say this about Harvard. This is a problem at every school in the Ivy Plus or among selective schools, so it’s not unique to Harvard at all. But they’re basically, I think what became clear to me, Andrey, is that you would think if you give everyone As, it takes the pressure off, but actually the opposite is true because students feel like they have to be perfect and they also feel like an A is not good enough, so I got to do a thousand other things.
And so employers were really saying, “Look, this is going to help us differentiate, and now we don’t need to ask for students to jump through a zillion hoops. The A means something again.” So that’s how they looked at it. I was actually surprised it was that high. I thought it would be more of a mixed bag, and that was quite influential to me in my own support of it was like, if employers are saying we need this, then I take that seriously.
Andrey: One of the things I’d be worried about is, yeah, employers can say that, but then when they’re comparing a Harvard and MIT student, or I mean, MIT doesn’t have as much grade inflation as Harvard, but a Yale student. The Yale students still have all As, and then if they don’t exactly adjust for the signal of the grade. I know a university that’s elite and has very little grade inflation is Caltech, and sometimes the students are very unhappy about how they get compared to other universities. [laughs]
David: Yeah, so I’d say two things about that. One is, you’re right, but that’s the kind of logic that gets you to never solving the problem. Because somebody has to go first. And actually, it’s an antitrust violation for us to coordinate on this together. So somebody has to be the first mover. That’s true. You laugh, but it’s true.
Andrey: Yeah, no, I know. It is.
David: We actually can’t.
Andrey: It’s true, yeah.
David: It’s non-price competition. Somebody has to do this, and so we just figured let’s leap first. And I hope that other, I mean, we didn’t coordinate, but I hope that other schools will follow our lead and go together. I think the other thing is, and Jason Furman made this point very well, is that if you’re worried about employer perceptions, you actually want a big change, not a small change, because then it’s salient to them. So we will write on the transcript, “In this year, we changed our grading structure,” and everybody will understand that, and that actually will mean something. Whereas if you just invisibly decrease the share of As by a few percentage points a year, you’re actually harming students more because people aren’t aware of it. So we kind of had to do this. So we had a very intentional strategy of big PR campaign, get it written up everywhere, get everybody talking about it so that when it happens, employers will know. And I think we accomplished that. Everybody and their mother wrote about it. I think there was five articles about grade inflation in the Wall Street Journal last year or something, you know.
Lightning round 52:41
Seth: Lightning round. All right. In the World of Darkness role-playing system, social attributes are divided into three different categories. Charisma, which represents social power, manipulation, which represents social finesse, and composure, which represents social defense. If you were starting a new game plus in your life, how would you allocate attribute points across those attributes of social skill? Bonus points if you give me the pre-AI answer versus the post-AI answer.
David: Defense would be my number. I’d probably do 50% on defense, because I think it’s important to be able to think for oneself. So to me, that’d be the most important thing. I think charisma’s very important, so I’d probably go 35 to 40% charisma, and I’d take the rest for whatever, manipulation, which sounds kind of negative to me, but I don’t know. I’m sure it’s valuable.
Seth: Charisma is just like your brute force. That’s how people are paying attention to you. You have their attention, and then manipulation, you can be a little bit more subtle about how you affect them.
David: I’d go 10 manipulation, 40 charisma, 50 defense.
Seth: I love that answer. Does AI change that answer?
David: Yeah, I’m thinking about that. So I’m thinking about it on the fly, live. I’m doing a posterior updating right now. I don’t know if I would change it with AI, actually. It’s kind of hard because I’m in the AI world now. There’s a question of what I would have done five years ago. I’m not sure. I don’t think I would change it. Maybe that’s disappointing to you, but would you change yours?
Seth: No, you took that question remarkably seriously. I really appreciate it. [laughs]
David: I try to understand the assignment, you know?
Andrey: So, you’ve done a lot of descriptive work now on AI diffusion, both independently and with teams at OpenAI. There are a lot of results there. So what we wanted to ask you is, what surprised you the most in this work?
David: One thing that surprised me a lot actually was in the ChatGPT paper, we did this analysis of people’s conversations for different topics, and we mapped it to O*NET work tasks, and we knew something about people’s occupations. So the question is, do people who are doctors use AI differently than people who are lawyers or medical assistants or teachers or whatever, software developers? And I thought the answer would be yes, people are using it for their work, and they are using it for their work, but actually the top tasks are the same for all those jobs. And the way I would put it is AI use is broad but shallow. So it’s like everyone’s kind of using it to manipulate information, like summarize, synthesize, understand, write, and for decision support, like asking for advice and what should I do with X, or how do I fix my dishwasher, or give me advice about how to deal with a coworker. And it’s like you would think that if you’re a teacher, you use it to educate people, and if you’re a banker, you use it to do financial calculations, and that’s just not true.
People are basically using it for the same things, but those things are broadly useful in every job. So I found that to actually be a quite hopeful story about whether this technology... And I mean, agents are different. This was pre-agents, this was chatbots. But it’s like there’s no way something like that is going to replace people because it’s so shallow. So I kind of updated positively in the direction of augmentation based on that evidence, and I had not expected it.
Andrey: Do you think that a lot of those questions sound related to social situations? So do you think that GenAI is augmenting people’s social skills?
Seth: Or substituting.
Andrey: Yeah, or substituting. [laughs]
David: I think it’s both. I think it’s doing both depending on the context. I think AI is pretty good for generic social advice. So, the AI is going to tell you, if you ask it, “Help me compose an email to my boss,” it’s like it’ll give you some general tips on how to do that well, but it doesn’t know whether your boss is a jerk or whether they’re sympathetic or how much your boss knows about you and the prior history. So, I think there’s a lot of context in social relationships, and so AI is pretty good at giving you a baseline generic boost, but it’s actually not very good at going one level up or filling in the gaps because you have a lot of... Part of it is the limitation of the user. It’s like if you don’t provide it with a context, it’s not going to give you what you need. And so even if the AI is super intelligent, it can’t read your mind, you know? And so I think that’s actually a human limitation, not an AI one, but it’s an important one.
Seth: My follow-up question is, what do we not know from all of these sort of descriptive papers about how people use AI that you would want to know in a next paper?
David: There’s a lot we don’t know. There’s much more that we don’t know than we do.
Seth: What’s one thing we want to know that would help us make some useful policy?
David: Yeah, so I think people are very focused on which jobs are going to be hurt and helped by AI and which are going to disappear, and they use these exposure indices and things like that. And that’s valuable work, but I think there’s a kind of not quite macro mezzo level of this that is extremely important, which is how AI changes the organization of work. Not just who’s going to do what, but like the Luis Garicano and other work on knowledge hierarchies, like firms organize like pyramids where there’s a lot of people at the bottom and they go, “Is AI going to change that?” My instinct is it will change that. But you don’t get that from an individual survey, and you also don’t get that from a big macro look at AI. You have to actually get inside a company and ask, how is it changing when it starts to use AI and how do roles change, how do functions change? And I’ve seen some work on that, but I think there’s way more yet to be done, and my prior is that actually that’ll be the big thing that AI does. I think there’s a good chance AI will change the organization of the firm in a fundamental way.
Seth: I believe that.
Andrey: Yeah, we do as well. [laughs]
Seth: We definitely do. David, I believe the last conversation we had was long before your days as Harvard dean. It was around 2018, we had a conversation outside of MIT where we were talking about sort of blind spots in economics. And you told me that one thing, correct me if I misremember, that you thought of the way that economists think about price gouging and price caps during crises as being sort of illustrative of economic logic breaking down, or maybe a better way to put this is economists not having good social skills. Do you still think about limitations on price caps or hoarding and price gouging during crises as an example of something that economists get wrong? And maybe have you thought of any new blind spots you’ve discovered in economists’ thinking?
David: That’s a good question, Seth. I don’t know if I’d fully agree with my past self. In the sense that I do think there’s some blind spots, but I’m not sure that’s the number one thing I would pick for sure, or even the number two or three thing. I think it’s more, I don’t know exactly what I said, but what I would say now anyway is an argument for why economists... We can write it down in an economic model and understand the empirical impacts of this and how, for example, caps restrict supply, gouging is necessary to clear the market, and all that stuff. But I think the equilibrium impacts of that kind of behavior and that kind of thinking are often negative enough to offset whatever efficiency gains you get in the short run. So it’s like, I don’t know if this is what I said, but a lot of times there’s a political dimension to economic choices that are hard to model explicitly but actually really matter in the way people perceive whether these things are fair or not. So, there’s a kind of- … first best outcome you can write down, and there’s a second best outcome that is like the outcome of a negotiation that can end up leaving society better off. I don’t know if I said that.
Seth: No, you said something very similar, and then I said- … “Well, if you allow people to gouge, then they’ll hoard in advance, and then there’ll be greater supplies if they know they can have a high price later.” And then you said, “Well, but there’s all these other bad general equilibrium effects,” right? But are those general equilibrium effects, are those really economic effects, or does that have to do with trust and relationships?
David: Well, I would say those are economic effects. Trust and relationships are. Here’s a line of research that I think is highly rated but still underrated, which is the work of like Bob Gibbons and Rebecca Henderson and others on relational contracts, because I think relational contracts basically run the world. You can’t really write a contract between people that specifies all the things that could go wrong or that could go right. And so a reason, and this is why trust is important, is it’s like, by the letter of this contract, I could screw Seth and Andrey, but I don’t want to because I value my relationship with them. And when that breaks down, a lot of bad things start to happen because... And you see this with, I don’t know, the changes in what’s happening in government or what’s happening in American society, where it’s a lot of the things we thought of as being like hard rules are actually just norms that you could break, and then they’re broken. And that’s the breakdown of relational contracts. So anyway, I think that’s great work. Again, highly rated. It’s not like nobody knows about it, but still underrated.
Seth: Well put.
Andrey: One final question, and then we’ll let you go.
Seth: To your very important job of, presumably bossing Harvard undergrads around and- [laughs]
David: I don’t do much bossing around. [all laugh] That’s how I remember my undergrad, was the dean was always bossing. [all laugh]
Seth: Putting people on double secret probation. [laughs]
Andrey: Who is the best teacher you ever had and why?
David: What a great question. I think Larry Katz is the best teacher I ever had. I don’t think it’s because he has the highest charisma lecturing. He doesn’t. He doesn’t, actually. I don’t think he would mind me saying that. We used to call him Encyclopedia Katz because he just has this incredible memory of papers. And so, when he lectures, it’s like drinking from a fire hose. It’s like this paper and this paper. But the thing about Larry is that I learned from the way he works more than just the lectures and the class itself, which is there are some scholars who do work that closes things down. So, there’s people who have a sense of intellectual closure, like this way of thinking is the right way and your way is the wrong way, and they tend to try to draw battle lines. And Larry is the opposite. He’s the most intellectually generous person I’ve ever met. He’s like, “I know all this stuff, but hey, if you have a new way to think about it, let’s talk about that.” And learning from him, I think it gave me a sense of openness.
I have one way of thinking about this world, but it’s not necessarily the right way, and I’m always open to other. And also a kind of openness to tools. So, if you need to write a structural model to understand something, do that. If you need to do an experiment, do that. If you need to write a descriptive paper, do that. Say, “I’m interested in answering questions. What’s the right tool for the question?” Rather than like, “I want to use this tool. Let me find something to use it on.” I also learned that from Larry. And in a lot of ways, I’ve told this story before. I told this at his festschrift. I came to Harvard kind of through the back door. I got my PhD in public policy, and I didn’t have any of the math background that one would normally have, and I really struggled in my first year of classes. At the time, I thought I was dumb. But actually, now I know people had seen this stuff before, and it just took me longer to catch up, and I eventually did.
But it didn’t feel great. And it wasn’t until I had the experience of working with Larry, and I basically wrote this paper in a labor class, which later became the first paper I ever published. It was about Head Start. And he sent me an email around Christmas time. I’ll never forget. And it was like a seven-page email. “Here’s all the things you could do in your paper.” At the very end, he was like, “I think this has potential to be published.” And I was like, “Oh my God, I can do this.” And the amount of time this, and there was 60 people in the class. The amount of time that he spent on this and on everybody else, I’m sure he did that for everybody else too, was incredible. And there’s just no better public goods provider in the profession than Larry Katz, in my opinion. And so I learned so much from that. And you guys have asked me a lot of questions about being a dean, and yeah, I’m a dean, I’m a suit, whatever.
In my heart, in my soul, I still identify as a researcher or a teacher. I got into it for that. But I also, when I took this job, I felt like I couldn’t say no because the institution’s been good to me, and I believe in what we do. And if you get called to serve, you serve. And it’s people like Larry who made me feel that way. It’s like this is actually an enterprise, and protecting it is worth doing, even if it’s kind of annoying sometimes and you have to have a lot of meetings. So, Larry was my best teacher.
Andrey: Wow, that was an amazing answer. Thank you so much. We’ve had so much fun. This is like a really-
David: Yeah, it was fun
Andrey: … meaty podcast, so thank you.
David: You guys asked the tough ones. That’s good.
Seth: Trying to keep you honest. And to our listeners out there, keep your posteriors justified.
Andrey: Thank you for joining us for another episode of “Justified Posteriors.” As a reminder, we’ve registered all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out.