AI and its labor market effects in the knowledge economy
Artificial Intelligence and the Knowledge Economy by Ide and Talamas
0:00Seth: Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. This is Seth Benzell, ready to lead the punk rebellion against AI middle management, coming to you from Chapman University in sunny Southern California.
Andrey: And this is Andrey Fradkin, managing by exception long before AI was happening. Coming to you from San Francisco. All right. What are we, what are we talking about today, Seth?
Seth: So I think it's a question that's a little bit different than we've discussed before. We are always obsessed with is this question of how is AI gonna plug into the social production function. What will it unlock? What will it help? Who might it hinder? And today we're coming at it from, a little bit of a different direction than before. Previously, we've kind of talked about, maybe some kinds of skills or tasks might be more substitutable or more complementary. Here, we might end up in the same place in terms of complementarities or what matters, but we're asking about, is AI gonna be our worker? Is AI gonna be our manager? Or is AI gonna be an expert, right? I think those are kind of three modes of interaction with AI that I'd love to know where we're gonna end up at. And we read in a very interesting paper that is trying to think through theoretically what those different worlds look like.
Andrey: Yeah. So just to kind of reframe that a little bit is we're thinking about a new model of organizations with AI. That, that's kind of what this paper is all about. We might as well introduce the paper. It's called Artificial Intelligence and the Knowledge Economy. It's by Enrique Ide and Eduard Talamas.
Seth: I have a question for you, Andrey, which is, all else equal, you can only have one of the three. Would you prefer to have an AI manager, an AI worker, or an AI expert to help you? So, maybe the worker is, can do more stuff at, a lower price point but maybe can't do the hardest stuff, whereas maybe you could have an expert who could help you kind of more theoretically, but with just, a light touch. This is, an Oracle AI as opposed to, an agent robot worker AI. Versus a manager. Maybe you would love to have the perfect AI boss that understands how to optimize all of your behaviors and gives you feedback. Which of those three visions for AI is the most exciting for you?
Andrey: From the economy point of view or from a personal-
Seth: Well, say give me Andrey Fradkin's point of view, and then give me the macroeconomics.
Andrey: Well, I don't, I don't want anyone managing me. I think I've lived my life to minimize the amount of managers I have. So I've... Certainly not that one. [laughs] Uh-
3:10Seth: So would you rather have, a tireless worker or would you rather have an oracle who can help you with a rare, difficult problem?
Andrey: This is, a knowledge worker that kind of does my coding- ... For me? Yeah. I'd rather have the worker. I think in the end the work needs to get done. And I mean... And so someone needs to do it, and I'd rather not be doing the tedious work involved. But from a society point of view, obviously, it has to be one of the others, right?
Seth: [laughs] Explain. Well, yes, that was exactly my intuition too. Right? Which is that, intuitively we all want to boss around AI serfs doing our bidding, but when you look at, A, what's actually the constraint on output of the economy and sort of B, where the highest marginal product tasks are, or I guess those are kind of the same thing, it really isn't in ordinary workers. Which, and again, if you increase the supply of these guys, you might drive down their wages, exacerbating inequality. If you want to unlock growth and lower inequality, you need AI experts and AI managers. And that's... I wonder if that's not just, a tension [laughs] that society is gonna have to deal with in the next 10 years.
Andrey: It's definitely, it's definitely a tension, right? What is so special about cognitive work anyway when AI's already can seemingly do large chunks of it. Certainly they can do the cognitive work of my students' homework very well. Not perfectly, but pretty... Better than my students. [laughs]
Seth: Right. Okay. And your students are presumably vertically differentiated?
Andrey: From each other? Or- Sure. Sure they do.
Seth: That there are some who are better and some who are worse. And if you were gonna plug them into a firm... Well, now I'm starting to kind of lay out the model that we're about to consider.
Andrey: Well-
Seth: But maybe before we do-
Andrey: Let's talk about our priors before we talk about- ... The model in particular.
Seth: Okay. Awesome. Okay. So it's kind of a vague question, like do we want an AI manager? Do we want an AI expert? Do we want an AI worker? Let's try to operationalize that into, rather than a desire, a prediction. So here's one prediction I'd like you to think about, Andrey. Within five years, let's do two cutoffs. Within five years, 10% of US workers will have managing or creating or deploying teams of AI agents as their main job. Give me the percentage chance of 10% of workers having that as a main job, and then give me, for the same five years, 2% of workers having that as their main job.
Andrey: All right. So I'll start with 2% 'cause obviously that's gonna be the higher number. 65%.
Seth: 65%, you think we're on path for that? That's my sense too.
Andrey: Yeah, at least.
Seth: I put that around 60%.
Andrey: At least. And then-
Seth: And that's mostly coders, that's mostly people who have really hands-on coding techie jobs, right?
6:14Andrey: Not only that. A lot of people spend their days interacting with B2B SaaS software, and I imagine- ... A lot of that will be agentized pretty quickly.
Seth: Right. Right. Obviously, LLM tools are really important and useful in all of these systems, but the question is to what how much of that can be delegated to multi-step agent processes, right?
Andrey: Well, like receipts. Let's do receipts and invoicing. I, our department has an admin. One of the things she does is track down every single payment made on the department card, gets receipts, puts them into some system, presumably manually. I can't imagine that being a non-agent process five years from now.
Seth: Okay, but now you for that... Okay. But for, now you need the second part of- ... The scenario, which is that she still has a job, [laughs] right?
Andrey: That's right. So I... Yeah, I mean, she still has a job for the reason that communication with everyone else might be something that requires a little bit of a human touch, and I have a hard time imagining the university, being so forward as to get rid of that. But, we'll see.
Seth: Well, you could imagine a universe where instead of having, this kind of more secretarial role, we have a vice president of, sales operations or a cost operations who is got one AI agent that's doing the receipt handling and one AI agent that's doing some sort of scheduling task. And, you can imagine a manager taking on many of these previously separate administrative tasks and delegating them to AI agents.
Andrey: Yeah, and but the same person does scheduling. I guess I'm just saying that I don't know to what extent people will be fired. Some people will be fired for this, but I think a lot of people will keep their jobs, but it'll just look quite differently. I don't know if that's any different than what you're saying.
Seth: Right. Well, there's... Yeah, there's a couple ways this could work. You could imagine creating a new job, and that new job is, agent wrangler, and that involves, firing people and retraining and reallocation across the economy. That's kind of, the more dramatic vision, which I think is still feasible. And then there's the kind of more modest vision where everyone's kind of quiet quitting by slowly offloading more and more of their stuff to their AI agents, and, it's out of, and they never substitute that time back into doing something else. [laughs] Or maybe they're doing quality improvement- ... So they're handling the most extreme cases, right?
Andrey: Yeah. And I mean, and I, that's why I kind of mentioned the university as a prototypical employer in that, it's not very good at firing people. It, people hang around even long past their useful, date. And so, yes, there would be some quiet quitting. But I do imagine that there are other things that admins can be, they can be more like concierges in the, in the future, right? Like- ... Like, I'd say, for example, other universities have an admin per faculty or per two faculty. We have an admin per, 14 faculty, right? There's plenty of things for someone like this to do, so that's what, why I don't think they'll all be fired right away. That, that's, that's all I was saying. All right. But let's do the 10% of the US workers. 10% of the- ... US workers, now that gets a little harder, right? Because, we're now reaching a point where we're getting into some non-traditional... Not the not the jobs that are traditionally spending their days interacting with computers, right? And so it's possible, but I would put the probability there lower, maybe at around 25%.
10:09Seth: 25%. I'm even coming in lower than that. I would say about 10%. I think there's something, agent management seems like actually a pretty sophisticated task versus set up automated system and ignore it, right? So I definitely think that this will be a job description, and maybe if you were asking me, 50 years from now, where are we, maybe that is, 20, 30% of the economy is managing these AIs. In the same way that, a lot of our job today is interacting with technology. I wouldn't describe, my job as operating a keyboard, [laughs] right? But it might kind of look like my job is operating a keyboard to someone from the 1800s, right? So there's also the kind of this, feeling question of when our job is managing AI workers, will it feel like that, or will they all be moving silently in the background and we'll feel like we're just naturally handling exceptions and we're not interacting with those guys? It comes down to, are you managing these things, or are you part of a system where this is automated by agents sometimes?
Andrey: Yeah. I guess another thing I'd say is there are plenty of, small business owners or solo entrepreneurs in the US, right? And I have a hard time imagining them not managing AI agents, right? Some of them already manage contractors, but, so there are so many tasks involved in running a business, and to the extent that we still have these people around, they will be using AI agents all the time.
Seth: Yeah. And I think you know, I think there will be a whole... This is kind of a secondary hypothesis about whether we think the long tail of, weirdos and small firms will be getting more or less important in the long run. But- You imagine that some of that long tail is only gonna be possible because it's like one guy made an entire movie from his living room, by putting together teams of AI systems and, some of them might be AI agents, some of them might be their own direct creativity. But really kind of making what we would think of as more of a production role in a movie, even if it's a movie of one rather than a, you know [chuckles]- ... Producer role, I should say. Yeah. So, yeah.
Andrey: But we can also think about, Uber drivers, right? Uber drivers are, quintessential small entrepreneur types. I know self-driving cars, et cetera, but anyway, we're talking about five years, so I don't think they're all gonna be eliminated. And they have expenses. They have to file all sorts of things, and I guess Uber can help them out, but, there's another world in which they're just gonna have, an agent deal with all the mundane things that are involved with being an Uber driver, right?
Seth: Well, I'm kind of abus-... Well, so it's, [chuckles] I have two thoughts immediately. The first is I would give Uber drivers as an example of the reverse. It feels like a, Uber is an example of the AI managing the human, right? That's a perfect example of high-frequency AI feedback that communicates exactly how well you're doing and exactly how much you're getting paid, and is, basically making algorithmic firing decisions. Isn't Uber the perfect example of-
13:31Andrey: So-
Seth: ... Being managed by an AI? [chuckles]
Andrey: So Seth, I mean, this so let's, we need to like, tell, talk about how the paper works, but I'll say that- ... Because you're assuming that the audience knows what the paper is. Um-
Seth: Oh, I wasn't. [laughs]
Andrey: No, no, I don't think your question makes too much, too much sense yet.
Seth: Okay, go ahead. Go ahead.
Andrey: I mean- ... It's very clear to me that Uber drivers are entrepreneurs, and they have to hire, or do various tasks that are involved with running a small business, all the mundane-
Seth: I think it's better, it's better to say the model doesn't really contemplate Uber drivers- ... Too well. Okay, so what-
Andrey: Yeah. So all-
Seth: ... So hold the Uber driver thought
Andrey: ... No one, no, factory workers truly, everything's kind of, you know... They're just workers, right? But I think a lot of jobs are obviously intermediate. They're not just workers, and they're not just managers, or they're not just, owners, if you will. Um-
Seth: Well, and this is maybe a way for us to talk about it later, is but they're horizontally differentiated. This is a vision of workers as vertically differentiated, as better and worse, when it's like, you know, sometimes the basketball player is taller and sometimes they're faster.
Andrey: Right. Yeah. So let's go to the other prior. The other-
Seth: Okay, other prior. [chuckles]
Andrey: The other prior is that LLM-based agents will exacerbate income polarization versus a counterfactual where they don't exist but other technology moves forward. And, just to bring up this prior, I think if we forgot about the past five years, the traditional story in economics is the race between education and technology. So essentially technology, increases the returns to high human capital, but at the same time, people are getting more and more educated over time. And so this creates kind of the two main forces that drive income polarization, toward, at this point, the gains in education or at least the share of the population that has a college degree or master's degree, it's kind of leveling off. So the prediction would've been that we get to a more and more unequal world as there are unequal returns to human capital. And now the question is, do AI agents kind of reverse that trend or not, or in what margins? So what do you think, Seth?
Seth: So I've been thinking really hard about this question, right? Because there is a lot of kind of emerging experimental evidence on does AI increase or decrease inequality amongst workers, right? We've, friend of the show, Erik Brynjolfsson, and, oh, help me out, who's co-authored on the paper, the call center paper.
Andrey: Danielle Li and Lindsay Raymond.
Seth: Yes. Also friends of the show, Danielle Li and Lindsay Raymond have an excellent paper looking at call center workers and showing, well, actually inequality and productivity goes down as a result of AI because it helps the worst workers solve the easiest problems. So that'd be a story in which if you wanna go from that story to the economy as a whole, that's a story in which AI is bringing up the bottom by helping the worst performers master the you know, the most basic tasks. You might think that that's just a partial equilibrium result. You might be worried, well, sure, in the short term, this is a tool that's good enough to boost the bottom up to the middle. But in the long term, isn't this a tool that just eliminates the bottom? So that's kind of the back and forth that's going on in my mind. I do think that there's some sense in which labor always has to be reintroduced, so it's not like you can't, [chuckles] it's not like you can kill the bottom, right?
There's no gonna be... I don't believe in long-term structural unemployment from technology. I think if you look around the world, people work approximately the same amount, high wage, low wage. And so therefore, I do think this polarization change, it could reduce polarization. I, that's kind of my instinct. I think if you look at the economy as a whole, you do see, I think people are more vertically differentiated than horizontally differentiated. In other words, I think the way, the reason you would get polarization due to AI is because AI is good at automating either people in the, people who tend to have, middle wages, right? But I don't know. I think that if you help people at the bottom, you decrease polarization. Went a little bit in a circle, but I end up at 25%. I think more likely than not, it reduces polarization. If we're talking about-
18:27Andrey: So you're at 75%.
Seth: I'm at... No, I'm at... Well, there's a po-
Andrey: That it can-
Seth: That it will exacerbate? I'm at... No, I'm at-
Andrey: Oh, yeah. You're at 25% that it won't exacerbate. Okay.
Seth: I'm at I'm at 25% that it will exacerbate, 75% that it won't exacerbate. That's kind of my theoretical prediction. That seems to be what's coming through in the studies we have available right now. Maybe 25%'s a little bit too strong, given that this is pretty preliminary, and we obviously have a story in which AI can boost inequality. Maybe it's gonna boost inequality across firms. But maybe let me... I, that's my sense, is that you're gonna boost up the bottom. That seems to be where the evidence is right now. Andrey, tell me why I'm wrong.
Andrey: [chuckles] Well, you're... Yeah. That seems overconfident. I... The way I think about this is that I don't trust any of these partial equilibrium results to begin with. They just don't tell you. It's not even about the call center workers being automated. It's just that the entire production structure of the economy is gonna change, and who's gonna get the rents in that is very hard to predict. So what am, what am I thinking about here in terms of polarization specifically? One thing is that we have a lot of highly paid jobs in the economy, which are essentially rent-seeking jobs. So thinking about lawyers and the like. It's essentially there are structural constraints to, there being competition in the industry. Now, there's a world in which AI agents break through that. So then I expect a reduction in polarization. Mostly because these people had just undue rents. Now, on the other hand, and this is kind of... This gets to the question of where, what type of inequality we're talking about. I kind of see a lot more opportunity for entrepreneurial types to create valuable goods and services through making their own companies or through joining smaller companies where- ... Which are managing lots of agents. Now, what share of that income is labor income versus capital income is a very interesting question.
Seth: Right. Yeah. We should I was evaluating this as wage polarization. And so I think that's how you were thinking about it, too, but you're absolutely right. If all of these extreme wins from blockbuster AI-driven companies, is that going to labor or capital? If it goes to capital, I'm more confident in my prediction that we're gonna see wage depolarization.
Andrey: Yeah. So, and then, and then I guess, it's not like inequality is... Certainly people summarize it by one number, but there's different parts of the distribution. I have, I have a hard time see... Thinking about, the top 1% losing wages as a result of this. You have to come up with, a lot of industry disruption for that to happen. I think that's unlikely. I think they will likely gain. They'll figure out a way to gain from these AI agents. Um- ... And the rest of, the distribution, it's really hard for me to predict because we have all these countervailing forces, including that, certainly low-wage cognitive, labor will get a lot of competition. So I would say my bet is that 50, 55% of... Hmm. My bet is that with 55% that it exacerbates it, just because I'm following the trend, which is that technology tends to exacerbate the polarization. But I'm very unconfident about this.
22:14Seth: Right. And then so you buy the Piketty story where inequality only goes down if there's a war [laughs] that blows up all the rich people. [laughs]
Andrey: No, no, no. I mean, if Inequality would go down, for example, wage inequality would go down if we had low productivity growth, for example, right? Where does income inequality come from in my mind? It's that when there are, high returns to doing things. So if there are no returns to doing things, then maybe we don't have a increase.
Seth: So somewhat a split in opinions going into the reading. I'm excited to see where we go. I think one just kind of side idea I want to bring up is I have noticed the stylized fact which it seems that human managers and managerial skills have been increasing in demand. And, I don't have a good sense of whether as LLMs get more agenty, to what extent that substitutes for having agency versus complements people who already have agency. San Francisco is convinced that it complements people who have agency. But I guess maybe it depends how much agency you have. [laughs]
Andrey: Depends on, yeah, we get truly autonomous AI beings versus just agents we call via API that are kind of, you know... Anyway, let's talk about the paper.
Seth: Let's talk about the paper. All right. "AI and the Knowledge Economy" by Enrique Ide and Edward Tlamas at the Journal of Political Economy, as I just learned. It's a theoretical paper. It's a pure theory model. And I appreciate just the kind of the comprehensiveness of the vision of the model. It's a pretty... The setup is very intuitive. There are two kinds. There are firms that are up to two levels. Either... So okay, so what do firms do in this model? Firms, they're, they are faced with problems in the knowledge economy. These are knowledge economy firms. They're faced with a problem And then the first decision is, what kind of hierarchy do I want to have? Do I want to just have just a worker try to solve the problem, just an AI try to solve the problem, a human worker with a human boss/expert assistant, a... And then every permutation thereof, right? [chuckles] So, worker, robot worker, robot boss, etc., right? Okay. So you make that decision, and then in order to make output, when the company is faced with a problem, first the worker tries to solve the problem.
If there's only a worker, that's the end of it. The worker gets paid according to the probability with which he solves the problem. But if you have a two-layer firm and the worker fails to solve the problem at the first stage, the worker can bounce it up a level to either an AI manager or an AI... Or a human manager that then gets a second crack at the apple, second bite at the apple to solve the problem. So this question, this paper kind of asks, given that setup, what kind of arrangements are going to be efficient as AI gets more productive?
25:21Andrey: Yeah. So that's a good description. Let me kind of just give some context. I view these types of papers as kind of intuition pumps more than anything else. Right? 'Cause it's very clearly, not the fully, full fidelity model of how organizations work. It's kind of-
Seth: Right. Stylized
Andrey: ... It's kind of gesturing at what, a model of, a true detailed model of the firm would look like because there is a hierarchy, but it's still obviously not how... And it doesn't work how any firm I know works, right?
Seth: Make a video game. Okay. You build, send it to your boss. He has to make the video game now, I guess. [chuckles]
Andrey: Yeah. Yeah. Yeah. Yeah. That said, I think we want to contrast this setup with another setup that we've talked about in this podcast series and just the most common task-based model that is-
Seth: Task-based model. [chuckles]
Andrey: That is used to, that is used to think about AI, right? In that model, tasks are, partially substitutable, with each other, and there's one parameter that kind of governs it usually. And it's very... Essentially the functional form by which tasks are aggregated to output is very, very constrained. And it's constrained in ways we might think are quite problematic-
Seth: Well, CES, right?
Andrey: ... When we're thinking about, when we're... Yeah, CES, which we, when we think about how the economy's actually organized with firms, and these kind of hierarchies of collaboration and production.
Seth: Well, can I ask you that- Can I stop you there then? Because, I mean, it seems like the most natural. So just to talk about the task-based model. So in a typical task-based model, final output is the CES aggregate over all of these different tasks. If you try to answer the question, what is AI's effect in polarization in that model, the answer would be, well, what, is AI substitutable for middle skill people? Then it's polarizing, right? That would be how you would answer it in that, in that framework. Are you just saying that you would want to see like nested CESs? Like there's a CES for tasks aggregating to a job, and then there's a CES from jobs aggregating to a firm, and then there's a CES across firms. Because I think that's, I mean, it's implied in Acemoglu, Restrepo even if they never actually say it, I guess.
Andrey: It's not obvious to me that that structure would give you the same thing as this structure. This structure has a sense. There's some-
Seth: But different than that. Yes. This is different, right?
Andrey: Yeah. This structure has a, has a flavor of like having a matching model, right? We're thinking about- ... What types of people to match with what types of people or what types of AIs in a production function. And there-
Seth: The way I would put... Yeah. No, go ahead, finish
Andrey: ... And there we get, and there we kind of get things that are a lot more, I would say, non-linear, non-constant than a one parameter or let's say even a nested parameter CES model.
Seth: The way I think about the difference is less that one's better and one's worse. I think about this as this is a model that wants to think about people as being vertically differentiated, and the task-based model is fundamentally about wanting to think about people as horizontally differentiated.
28:32Andrey: Yeah. So I, so I never claimed that this is a better model.
Seth: [chuckles] Sorry, I didn't mean to say that.
Andrey: [chuckles] I guess I would say like the emphasis, you're right in the emphasis, but I still think that you can have a horizontal model where you have much more interesting production networks than the CES.
Seth: Right. Okay. So let's, yeah, so this is, this is gonna explicitly think about what that production hierarchy is gonna look like.
Andrey: Yeah. I'm gonna ask you a question about the setup, Seth.
Seth: [chuckles] Please.
Andrey: Because you keep using the word manager. Does a Does... What does a manager do?
Seth: What is a manager?
Andrey: Yeah. What does a manager do?
Seth: So in the, in... [fake sobbing] It's a good question, right? So in this paper, the paper refers to the people at the higher level of the company as solvers, which is not a position at any company I've ever heard of. It kind of seems like a conflation of two different things. It seems like they have a model of workers on the one hand, and then like experts/managers/exception handlers as the second type of job. It does, it conflates all those things. What do I think of as a manager? I think of a manager as someone who sets strategic direction, who shapes company culture, who makes hiring and firing decisions. What do I think of as an expert? I think as someone who can advise low-skill workers on exceptional cases and can bring, additional context that might be able to help them solve a problem that they were stuck on. We don't usually think Yeah. So I, the paper I think absolutely conflates these two because it talks about a hierarchy when it is really talking more about expertise rather than about managerness. Or how do you think about what the paper's doing?
Andrey: Yeah. Yeah, it's talking about consultants. That's how I think about it. Right?
Seth: It's bringing in consultants.
Andrey: It reminded me what I do, you know? So, [laughs] I get brought in, people ask me a really hard question, and I come up with an answer. But it's not, management when it's oftentimes you see a job posting for a manager, what are they gonna say? Leader, leadership. What is leadership? Leadership is the ability to interpersonally be connected to a bunch of people, right? To get them to all move, work together in a productive way. And that requires kind of building a team culture and- ... And so on. I just... And I think one of these things is actually a lot easier for AIs to do than the other, right? It's much easier for the AI to answer expert level questions than to build a team culture and be a leader.
Seth: I think that's probably right. I think you're, you, in order to talk about that, you need heterogeneous abilities, right? You need someone to be good at leadership and someone to be good at solving the object level problem, and this paper does not contemplate those being separate things.
31:40Andrey: Yes. I just wanted to flag that. But once again, we're viewing this as an intuition pump more than-
Seth: Right, it's an intuition pump. But I think that-
Andrey: We're not taking it super seriously. Yeah
Seth: ... I do think that's a key, but that's a key that's a key context, is that we're conflating experts and managers.
Andrey: Okay. So Seth, great. So without AIs, let me tell you kind of what happens in the model. We have... And let's talk about the version of the model where no one is kind of working for themselves, although there are situations where that might happen. So essentially, the highest, they're, based on skills, you're either gonna be a solver or a worker, and there's gonna be a cutoff point. So everyone with a skill greater than that cutoff point is gonna be a solver, everyone below is gonna be a worker. And also, people are assortatively matched. So that, that kind of means that the best people who are still workers are matched with the best managers. Or sorry, with the best solvers. And that's something that creates the most production, 'cause the model is solved in a way in which there's a perfectly competitive, economy, which is kind of, probably doesn't mean very much to people who are not economists. And then, and then we kind of go to the world with AI. Now how they're gonna think about it is that AI can do, it has a skill level, and all AIs have the same skill level, and that's because, you know-
Seth: There's only one AI. [laughs]
Andrey: There's, it's trivial to copy an AI. That's kind of their... They're definitely not thinking about the reasoning models here, right? They're not thinking about compute as something that is a flexible variable that solves harder or less hard problems. They're really thinking-
Seth: They're also not thinking... Yeah. And they're also not thinking about a price quality trade-off, which also is a very live issue.
Andrey: Yes. Yes. That is, that is very true. But anyway, so depending on the skill level of the AI, it's either gonna be a worker or a solver. And it's gonna affect the equilibrium. And I think kind of the meat of the, of the paper, like the paper does several things, but I think kind of the key thing of the paper is they're gonna draw essentially two different wage curves. One wage curve for when there's no AI, and one wage curve when there is AI, uh-
Seth: And then a third one when the AI isn't allowed to work without supervision. The silliest case. [laughs]
Andrey: Yes. But let's first focus on the, on the, on the on the, on the case without that. So- ... If the AI skill level is that of a worker, then we see an equilibrium where for people with the lowest skill level, their wages actually go down. For people who, have the highest skill levels, their wages go up. That- ... That, that's kind of the simple TLDR. Perhaps not surprising. They're not talking about robots as knowledge work, but imagine we had a bunch of robots that could automate, factory labor, then you would imagine that factory workers are gonna be hurt by this, and maybe everyone else is helped by this. Um-
34:42Seth: Right. This is the classic, it substitutes for low-skill work, and it's like implicitly a compliment to high-skilled workers because the high-skilled workers don't have to do the dumb shit anymore.
Andrey: Yeah. Yeah. Now if we have kind of very knowledgeable, very skilled AI labor, then we get that there's some amount of solvers who are actually hurt by this. But the best solvers are still helped, and then the workers are helped as well, and that's kind of because everyone's becoming more productive now we have all this additional, production capacity due to the AI. Yeah, so that's kind of the basic gist of it, and-
Seth: Right. I want to throw- ... One more intuition I think is good here. So one other intuition here is just that the very, very best expert is only ever gonna get helped by AI, because they're, that, that one last edge case that they can solve is just gonna get more and more valuable as AI solves the easier cases. That seems pretty plausible. [laughs]
Andrey: That, that's right. I mean, to me it just seems a little strange to think that AI is, it's, the one-shotness of these problems is a little weird, right? Like- ... We can like take 10 different tries to solve a really hard problem or, if the, if it's the Riemann hypothesis, then maybe a million tries. I don't know. Right? So it's, it, there's a sense in which they're not really seriously engaging with how much effort it takes to solve a problem. They're just assuming that every problem-
Seth: You don't wanna think about this as like a day. What about if you reinterpret this as like, you show up to the office that day and you either have a productive day or you don't have a productive day?
Andrey: I just don't think that that's That, it just doesn't seem true to how most, many many of the most important problems work. But anyway.
Seth: Right. So okay, so how would I think about solving the Riemann hypothesis in this model? Yeah, I mean, in this model, it's just you go to the most expert, and you pay them what it costs, right? [chuckles]
Andrey: Yeah. Yeah. It doesn't, there's no there's no probabilistic solving. It's just some problems are really hard, and only the smartest people can solve them. But that's not, also not true in the world, right? If you get enough not that smart people, they'll eventually solve the problem, too. I mean-
Seth: Right. That's not in here
Andrey: ... Or if it-
Seth: The associative matching.
Andrey: Yeah. Or-
Seth: There should be associative matching of firms to problems.
Andrey: Yeah, and then, and also people are better at solving problems in teams, even if they're not all, very smart, right? There's no- ... Sense in which this is true. Anyway, we're getting to kind of the limitations already. I guess, is there anything else you wanted to say about this paper? It does have some other side results, but is there anything else you want-
Seth: I think let's just, let's throw in that last result, which is, okay, so the result you, Andrey just gave the result for what they consider AI that's allowed to be, quote-unquote, "autonomous," so that is AI that is allowed to be a worker that's either managed by a human or managed by other AIs. The paper also considers a case which, Andrey, it's your sense that this is a Reviewer 2 [chuckles] that they added this case, 'cause it feels a little ad hoc, which is what if AI is banned from being a worker? It can only be a manager/expert/copilot. The term, the conflation gets a little bit even more extreme as we get to this phase. In that case, we, you kind of get the opposite result, right? If the AI is able to now boost the productivity of the worst workers by, advising them, et cetera, but isn't allowed to be a worker under the best workers, then you're gonna boost up the bottom, and I think they try to connect this to that Lindsay Raymond, Daniella Lee, Brynjolfsson set of results.
38:34Andrey: Yeah. Yeah. Yeah, it is, it is quite contrived, in my opinion, of a setup. I don't think, by the way, I don't think it's a Referee 2 at all.
Seth: No?
Andrey: I really don't. No. No. I think it's a little game that economists play where they wanna make their model seem relevant, and autonomy is, makes it more interesting, right? I mean-
Seth: It's a buzzword.
Andrey: Yeah. It's not a buzzword. I, obviously we think it's really important. But you have a model that's ostensibly, about AI, and you wanna incorporate autonomy into it, so you define it in the model in a way that no one else would ever think about defining it. And then you make claim, claims about what autonomy is gonna do. Um-
Seth: Right. There's a little, there's a little bit of a parlor game of, can you, can you find it- ... Under the hat, right? They're moving around the definitions, the goalposts on you. To a certain extent, good theory work should be about helping you build a better definition, and this seems to be just muddying the waters about these terms rather than clarifying these important-
Andrey: Yeah. To what- ... To what extent is a consultant... Yeah. The consulting AI versus a working AI would... I'm certainly using worker AIs to do some of my code right now. So- ... Just to, so for, in terms of this model, I don't know what to make of it, of this.
Seth: Well, obviously we're in the reality where worker AIs have not been banned. They're- [chuckles] They're... That's the Butlerian Jihad scenario- ... I guess. But it's, not even, because the AI's allowed to be your boss. It's so I, yeah, it's a little bit contrived- ... In my opinion.
Andrey: Yeah. So let me, let me talk a little bit about what I think, is another key issue in this paper for how- ... Seriously we should take it. And it's actually kind of why I did not take... Why I just simply don't take this paper seriously, actually.
Seth: Ooh.
Andrey: It assumes that there's an infinite set of problems and that they're equally valuable.
Seth: Right. You could think about this as a divisibility idea, right? That any problem is divisible into atoms. That might be all of this-
Andrey: No, that's wrong. That is not-
Seth: No?
Andrey: ... How I would interpret this paper at all. Um-
Seth: No
Andrey: ... Because, once again, if the problems were divisible, then each one of them would be worth less. But every problem here is worth exactly the same amount. The problem with this paper is that marginal problems are just as important as infra-marginal problems, and that is absolutely not true. And as a result, when we think about, even the basic results where, wages might go up for certain types of workers, that in no way takes into account that this marginal production is gonna be less valuable than infra-marginal production. I can imagine- ... Wages just going down across the board if marginal if marginal problems are not very important.
Seth: Right. So, okay, so let me try to wrap my head around this concern. So, the idea here is we should think about each firm as making a differentiated good, and maybe some of them false, solve problems that are on average easier, and some of them solve problems that are on average harder. If we get better at solving easy problems, that should disproportionately boost the productivity of the makes easy stuff firm, thereby lowering the price. That's kind of the argument?
41:53Andrey: Quite. This is not about hard or easy problems. This is... Because there are plenty of-
Seth: Well, a marginal isn't a marginal problem. That's the Riemann hypothesis. Or am I not, am I misunderstanding what the marginal problem is?
Andrey: So we can think about it that way, but I have something that's a little more basic. Forget about differentiated problems in terms of difficulty. There are difficult problems that no one cares about, and there are difficult problems that- ... That people care about. Currently, the economy is working on the difficult problems that more people care about, and now we have a bunch more production, and so now the problems that people are gonna be working on, even if they're very difficult, no one really cares about them, right? By the way, that might be the Riemann hypothesis[laughs] I don't, I don't know. But, to the extent that, some problems... Let's think about, the entrepreneurship framework. We can think about building robots for doing, cleaning the house, which a lot of people would care about. That could be pretty useful, right? And- ... And then we can think about robots that juggle. And clearly, robots that juggle are not as valuable to the economy as robots that clean the home, but those are both problems to be solved, right? There's no sense in which-
Seth: Well, I mean, okay. But that-
Andrey: Yeah. This is an-
Seth: I think that's an unfair... I don't think that's a fair critique, 'cause you can imagine that there is a every problem is in a place on difficulty, usefulness space. And when-
Andrey: There's no usefulness here in this model, period.
Seth: No, I'm saying... No, I'm saying... Yeah, I'm saying, okay, zoom out, and I'm gonna tell you why- ... This model captures that. All right? So imagine that there is this space of the difficulty of a problem against the usefulness of the problem. And you're right, there's a whole bunch of that space where no one's ever gonna do the job 'cause it's super unuseful and it's super hard. But there's gonna be, a Pareto possibilities frontier of the best problems to work on, right? It's gonna be the curve of problems- ... That have the right difficulty, usefulness trade-off.
Andrey: And now when you increase production levels, surely the usefulness has to go down on average.
Seth: Right. And then you would need there to be heterogeneous firms attacking heterogeneous problems, and then those, they're all producing at prices, right? That's how I would think about adding that in.
Andrey: Yeah, you can think about different prices, sure. But regardless, I mean, look, this There's a world where the robots can just make everything, right? And there's... That everything we currently need, let's say, and it, that is kind of not allowed for in this model because there's infinite things that are equally valuable to humans, which to me is a lot of the key reasons why we think there might be wage effects. And I'm specifically talking about the wage part of this. A lot of the key reasons we think there might be wage effects is this aggregate influx of workers taking away the useful problems. Sorry, that's, that was me going on a rant about this.
44:58Seth: No, that's good. It's good. I'm sure this is a topic we'll continue to revisit. Effective AI on inequality is just gonna be perennial. So if you like this content, audience, [laughs] let us know. [laughs] All right. I wanna... Before we move into maybe posteriors, I wanted to just bring up quickly a complimentary paper. Listeners might be interested, if they liked this paper, in the paper Generative AI and Organizational Structure in the Knowledge Economy. A very similar title, from Fasheng Xu, Jing Hu, Wei Chen, and Karen Xi that I recently saw at Dan Rock's Wharton AI Conference. The reason I bring it up is it's just kind of an interesting example of kind of convergent evolution in modeling, and also maybe a reason to kind of lower your confidence in the results of these models. Because I would say that they start with kind of a very similar setup, which is that there's a continuum of workers from, the most expert to the least expert workers.
The workers either solve the problem on their own or they can boost it up to either an AI expert or a human expert. It's got a lot of the same elements, and yet because they have slightly different assumptions about, the qualities of AIs, so for example, they distinguish between the level of problems AIs can attack and their hallucination rate at different levels of advancedness, they end up getting kind of, sort of an anything-goes result. So you might need more managers, you might need less managers. They do find... One interesting result is they do find a little bit of de-skilling. So if, so in the model we just considered, workers' skill is exogenous, I think, if I recall correctly. [laughs] Did I... Or no, no, people pay to get skills. I forget. In that-
Andrey: No, they don't. They don't.
Seth: They don't. It's exogenous skill. In the in the model I'm just bringing up, there's endogenous skill formation, and what they find is if workers are able to delegate some of the complicated situations to AIs, that can lead to de-skilling and workers acquiring less education. So, you can see how even if you start with a very similar premise, by giving the agents different affordances, you kind of get very different results.
Andrey: Yeah. But this is kind of why I'm not a huge fan of the approach for that paper. It's obvious that you can get anything goes. We've all taken- ... Micro theory. We know, we all... These are all... You make a model complicated enough, and you can allow for anything. And I think, like- ... Even though the paper we're currently discussing, I'm not super convinced by it, there... What's nice about it's very simple and it gets very stark implications. So it is an intuition problem. Once you build a complicated enough model with many moving pieces, all that kind of goes away. And I tried reading the abstract of the paper you're recommending- ... And it's unreadable. It is completely unreadable. Once it... What... In... Look, I don't, I don't blame the authors. They tried-
47:58Seth: Because it introduces a lot of terms that it needs, okay, like yeah, the, I mean, in the same way that the first paper uses autonomous and non-autonomous, and then you have to dig to page 30 to figure out what that means. These terms to credit, yeah.
Andrey: But then it's, way more in that direction 'cause they try to model a bunch more things in very ad hoc ways, right? So-
Seth: Right. They try to do more stuff, right? Because they try to do more stuff, you end up with more of these, y- ... Conflated ideas.
Andrey: Yeah. So kind of, when I think about economics papers, I like either papers that are quite simple but give a new intuition- ... Or, that have the minimum of subtle, minimal set of assumptions to give an intuition, or papers that are quantitative Where, because we know that more or less anything goes in a lot of ways. If we can tie things to the data, that at least tells us kind of which of the various forces in a model wins out or what are the magnitudes involved. But this middle ground of a complicated theory model where anything goes, to me, is not very useful. Um-
Seth: Right. And I'll say, but I'll say, I'll add one more thing, which is that I would have loved to see in this paper that we just read, the JP paper, like a little bit of, here's some theoretical predictions about, what you should see in different experiments. I feel like maybe I missed it, but they didn't take particularly strong strands on, this is the experiment that would either prove or disprove this theory. And I'd love to... I love when that's thrown into these things.
Andrey: Yeah. Yeah. I mean, I do think they speculate a lot in the conclusion, and I found those speculations very not convincing. So-
Seth: [laughs] All right. Any more limitations or other things you want to bring up? Are you ready to move into posteriors?
Andrey: Hmm. Yeah, I think, I think a final limitation, I don't, I don't think they needed to include this, but it should affect how much we learn from it, is that we're talking about a five-year span for our prior. Surely we need adjustment costs, right? Surely we need... We're not seamlessly moving from one perfectly competitive equilibrium to another, and that the transitional dynamics might have, might result in very different wage dynamics and employment dynamics than, what we see in this paper.
Seth: Right. So this is a model after everyone gets perfectly resorted into their firms of exactly the correct productivity. There is... There's not an adjustment cost. There is a cost of booping, bumping your problem up to the higher level. I know that's not what you had in mind.
Andrey: That's not it. That's not it.
Seth: But I guess if you had that bumping up cost be, decreasing over time for bumping up to AI, maybe that's how you'd [laughs] implement this adjustment function.
Andrey: No, I don't think so, because I think what's, what I'm envisioning is, hiring and firing costs. Costs of bringing- ... AI online, investment in, data centers and chips and so on. Firms that might not be willing, might not have enough competitive pressure to adjust because they're, they have market power, right? I mean, we have to be really careful with, thinking that we're gonna move from one competitive equilibrium to another in a very short time span- ... Without anything else happening.
51:18Seth: Right. Right, even if the technology was, perfectly... Even if the technology did get out to everyone, just to use the technology, that takes even longer to diffuse. Good point. All right. Let's time to justify your posterior. The first question, Andrey, we asked was, within five years, will greater than 10% and will greater than 2% of US workers have managing or creating or deploying teams of AI agents as their main job? When we talked earlier, I think you said a 65% chance of greater than 2%.
Andrey: For 2%.
Seth: Yeah. And then a 25% chance of greater than 10%. Have your beliefs moved?
Andrey: No.
Seth: No. [laughs] I came in with a 2% chance of greater than 60% and 10% chance of greater than 10%. My opinions have also not moved. [laughs] On the second one, though, maybe I've moved a little bit. How about you, Andrey? LLM agents based on LLM-based agents, AI agents, excuse me, will exacerbate income polarization versus a counterfactual where we get every other technology except for AI agent workers. You said 55% chance that this exacerbates income polarization. Did your view change?
Andrey: It didn't, and I really have to say that without this aggregate demand, aggregate diminishing returns to problems, I just... It's not a macro model, so how can I take it to be, to have macro predictions?
Seth: What you're calling... Yeah. The way I think about it is, either heterogeneity at the task level or heterogeneity at the product level, right? You need a little bit of that in order to have the real ju. Still, I would say over the course of this conversation maybe, in addition to reading the paper, I would move up from, 25% to maybe 30%. I think the reason... This paper tells a story in which the very top always has to benefit from better and better AIs just because AIs, up until they get to the point where they can fix, do everything, there's some sense in which that's kind of mechanically complementary to the last thing that they can't do. Andrey, on the, difficulty usefulness curve, right? [laughs] You know, taking that into account. So I move a little bit in that direction just because my intuition was pumped a little bit more just for that edge case.
Andrey: Yeah. And I guess, my model, I really like the O-ring theory of production and generally, I think that AIs, until they can truly do everything, which I think is gonna take a very long time, we're in an O-ringy world, and that last bit of human labor is actually gonna get a lot of benefit.
54:19Seth: Right. And so, and then the question is that very, very last guy, is he someone who is he or she, is he someone who's already at the 99th percentile, or is that someone at the 50th percentile- ... Or 25th percentile, right? Um-
Andrey: Well, yeah, but we don't know. But this is the thing is, ability is clearly, I think a lot of people would be tempted to put in human capital or- ... IQ or whatever, something like that in, as the, as a stand-in here, and it's not obvious to me. Maybe it's a very specialized skill that not, everyone has. For example, leadership.
Seth: Or maybe it's plumbing.
Andrey: Leadership.
Seth: Or maybe- [laughs]
Andrey: Or plumbing.
Seth: Right, leadership, right. Yeah. Yeah, exactly. Yeah. Maybe it turns out that, when we automate everything, the last job is plumbing, right? And so that, and that's a vision where the heterogeneity, the heterogeneity in... Sorry, the horizontal differentiation is so much more important than the vertical differentiation. Which is what this is a paper about.
Andrey: Yes. All right. Well, that was- ... A spirited discussion about a very-
Seth: It certainly was
Andrey: ... Theoretical paper, yes.
Seth: Oh, I wanna throw a... Before we close out, Andrey, I wanna throw do some shout-outs.
Andrey: Shout out. Shout out.
Seth: For, I'll do a shout-out for friend of the show, Sebastian Stephan, who, and Matt Bean, who have both sent on excellent comments about material that we should cover soon. We're taking a close look. I know there are many more who have also suggested stuff. When we choose your paper, we will talk about you on the show and probably say nice things, although it's hard for us.
Andrey: We can say nice things about our friends. Right?
Seth: Our fans. We can say nice things about our fans.
Andrey: Fans. You don't have to be our friend to be our fan. That's very important.
Seth: Very true.
Andrey: [laughs] We have the best fans. We have the best listenership of any podcast. I have no doubt about that.
Seth: Our beautiful, brilliant, enticing fan base, I dream about you. I hope you think about us.
Andrey: [laughs] On that note, thanks. Thanks for listening, and please make sure to like, comment, and subscribe.