EMERGENCY POD: Is AI already causing youth unemployment?
We discuss "Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence"
0:00Seth: Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzell, born too late to screw over millennials with government deficits, but just at the right time to screw over Generation Alpha with AI, coming to you from Chapman University in sunny Southern California.
Andrey: And I'm Andrey Fradkin, putting on my miner's hat as I run into the coal mine, finding those supposed canaries, coming to you from San Francisco.
Seth: You're looking for the, looking for the canaries? Oh, I thought you were gonna make, the Minecraft reference of the children yearn for the mines.
Andrey: The children have experienced many things.
Seth: The children love Minecraft. If Minecraft was a hobby, would you say?
Andrey: Well, it is, but these days the kids are into Roblox, Seth. I don't know if you got the memo.
Seth: They got... Why not, porque no los dos? The reason I'm asking you, Andrey, is Minecraft exposed to AI, and are the kids these, the kids these days interested in Minecraft is because we're reading something today about how does the rollout of AI technologies affect employment and occupational outcomes for youthful people? Which is really exciting given that youthful people will be paying my social security benefits with any luck.
Andrey: [laughs] That's a very selfish way of looking at it, Seth.
Seth: [laughs] I'm sure future generations also have intrinsic value. I say this as someone who has just reproduced, so you know. So yeah, first-
Andrey: Oh, tip of the cap. [laughs]
Seth: So this is, we are as a brief personal aside, this is the first episode we're recording post my paternity, and, I'm as obsessed with intergenerational, justice as, ever. [laughs] Can't say it's changing. And Andrey, not only can I not say that, what I also cannot say is that there is no emergency, because there is an emergency. This is our first emergency podcast as well. What's the emergency, Andrey?
Andrey: Well, listeners of the show and followers of the AI discourse will know that the public demands evidence that the AI is taking our jobs. But so far, the world has not provided said evidence. Um-
Seth: Unfortunately, reality has not complied. [laughs]
Andrey: Reality has been slow to comply. So obviously, we've been on the lookout, and I think everyone has been on the lookout for this first evidence that AI is doing something to the labor market. And recently, there have been two papers that purport to show that there are effects of AI on certain aspects of the labor market, and one in particular has gotten a lot of media attention by our post doctoral advisor, Erik Brynjolfsson, Bharat Chandar, and Ryuhyung Chen, called Canaries and Coal Mines: Six Facts About the Recent Employment Effects of Artificial Intelligence. A lot of our listeners and a lot of people around the world want someone to actually read this paper- ... Rather than just reading the abstract and publicizing it.
3:27Seth: Clearly, it's been the paper heard round the world, and if you're listening in, you're gonna get the inside scoop about what do we actually learn from this emergency dump of information.
Andrey: Yeah. So, as is our usual format, we should state our prior, Seth, isn't that right?
Seth: Like we said, we are talking today about is the AI taking our jobs, but specifically, is the AI taking those good Gen Alpha jobs, right? That, people graduate from college, they're ready for their new software engineering career, they're ready for their new customer service career or marketing management career, and snap, actually that entry level job doesn't exist anymore, Andrey. We're gonna give that to the AI. Or at least that's how the- ... It's most pejoratively framed. So the first question that this paper is speaking to is this idea that US entry level employment growth has been slower for the most AI exposed occupations since 2022, right? So is there something about the combination of young workers and AI exposed jobs that's leading to a depression in hiring? So how do you think about that?
Andrey: Yeah, I mean, just, to, think about this from first principles, it's not something that I spend my days, looking at these numbers, right? So is it plausible to me that this employment, could be decreasing? Sure. But also, is it plausible to me that it's not decreasing? Also could be possible. We had a pretty good economy, at least according to most metrics, especially the unemployment rate. So if we think that useful labor gets employed, then it seems that it would be hard to find something on the employment rate of the youth, since they have a pretty low unemployment rate. So let's do, I'm kind of... Obviously, we've read the paper, the usual conceit of this show, right? But we're slicing to a subset of jobs, right? And so even a big percentage of a subset of jobs might be a very tiny percentage for the overall employment rate of a particular demographic. So, I think I'd be surprised if it's more than an order of, 10%.
Seth: You're comparing, the most exposed jobs to the least exposed jobs. You would not be surprised if the most exposed jobs grew 10% slower for entry level workers.
Andrey: Yeah, I think plus or minus 10% would be plausible to me.
Seth: Oh, even minus 10% you wouldn't be surprised.
Andrey: Yeah, well, descriptive I mean, look, there's industry variation in hiring rates, and exposure is correlated with industry, and so Why not?
Seth: Why not? So-
Andrey: I'm taking the prior seriously.
Seth: No, no.
Andrey: What about yourself?
Seth: Let's bone the youth, 'cause I did not even try to attack the sort of magnitude question of how big the effect is. Because I agree with you that even signing this is a little bit complicated, right? We... What was the last month of discourse? The last month of discourse was how many hundreds of millions of dollars does it take to hire a 25-year-old superstar software engineer, right? So, the... It's not immediately obvious that the AI is reducing demand for skilled young people who work with computers, right? That being said, it's fair to say that those superstars are maybe not representative of the median person, or maybe the person who's in this data set, something to think about. The very top AI superstars are probably not in this data set. If we're thinking more like median people, I'm thinking about the story that we heard from another friend of the show, Matt Beane, whose book, The Skill Code, is a lot about how companies are struggling in this AI age to figure out what to do with their entry level workers.
Because if you can automate the simplest thing, and that was the thing that helped you onboard people, now you've got, a question of like, do I even wanna hire entry-level people? If I do hire entry-level people, is it gonna be even more emphasizing training, and it's gonna be an even longer training period before they're positively productive. And I'd say, on net, that story, I think, makes the most sense to me. And so I thought maybe 70% chance going in we would find a sign in the direction that the authors claim, that America as a whole, there really is some sort of difference here that the youth are doing poorly in AI exposed occupations. I'm not really thinking about software engineering, I'm thinking about customer service and certain kinds of management bureaucracy jobs.
8:07Andrey: So, Seth, are you already mixing in the descriptive and the causal?
Seth: Oh, did I? Did I say something causal right there?
Andrey: Well, you gave a story that's a causal story for why something's happening.
Seth: Well, causal story generates the description, right? You know?
Andrey: Well-
Seth: I think reality is caused by things.
Andrey: No, so to be clear, there are many other reasons, like I mentioned- ... Just why things fluctuate, right? So industries have ups and downs. We'll talk about some of these in the episode. So it's not-
Seth: There's a lot of confounders pushing in the-
Andrey: If you looked at, employment for the youth across sectors over time, you would see a lot of variability. And none of that has to do anything with AI. We'll see a lot of variability. So even though AI is a story in the mix, right, it's a separate question of whether that's the primary... If we do find this thing, is that the primary reason for it, or- ... Is it something else?
Seth: Very well put.
Andrey: So what do you think about that?
Seth: Very well put. I think let's save all the confounders we have in mind for a little bit later in the description. But yes- ... Absolutely one of the reasons that I'm putting such a high probability on finding this descriptive result is I also have some non-causal theories about why we would see it.
Andrey: I see. Yeah, and I guess what I would say is that it's a really interesting time to be running these sorts of studies and thinking about base rates. I think this goes really to my point about the demand for research. This is an obvious research paper in the sense that, I've had this idea. I've, I can give you the document that writes it up in-
Seth: We've seen people do it
Andrey: ... In March. So it's just a question of getting the right data, and that's one of the tricky things.
Seth: It's a very special data set, and we'll talk about it in a second.
Andrey: And you're looking at a slight... Yeah, you're looking at slices of subsets of people that might be hard to measure in certain data sets. And then there's obviously this other thing, is that there are many ways to skin a cat. You know? You can analyze a data set many ways, and we've seen headlines that there's been, as recently as a few months ago, there's no, there's no evidence that AI is affecting labor market. And now we see a paper that says that AI is affecting the labor market. So to me, I actually... My base my thing was it can't be that big of an effect, 'cause we would've seen it already, in a more obvious ways. And the, 100 different research teams that are writing the same exact paper, one of them would've, done it. So the question is, how quickly, how quickly did you expect this finding to come in from research teams?
Seth: Well, what's the twist?
Andrey: And maybe they're-
Seth: So why this result? Just mechanically. Mechanically they're able to find this result 'cause previous studies had mixed together age groups. Heard it w- [laughs]
Andrey: Well, let me push back against that.
Seth: Ooh.
Andrey: The previous studies, certainly have looked... Once again, this is a very obvious... Not in a bad way, this is such an obvious idea that anyone looking at this type of data would have looked at this.
Seth: This is the first thing you should do. [laughs]
Andrey: I think this is second thing you should do. The first thing is not split out by the groups. This is the second thing you should do.
Seth: But age, I think there was a, there was a little bit of an insight there to think, look at the age split.
Andrey: It's so obvious.
11:07Seth: I know you know. All right. [laughs]
Andrey: It's a... Read Matt Beane's book. I'm sure other people have looked at the split, probably with a data set that was maybe more noisy, so.
Seth: I'm not aware of a data... So the studies I know about are, industry level, firm level. The... I've never seen occupation times age before as a split on one of these.
Andrey: Well, I guess what I'd tell you is they probably looked at it was noisy- ... And then they decided not to put it in the paper. Just being honest about how research works in this field, right? So I guess what I'm trying to say is that it's actually interesting to think about if this were going on, how early would we be able to detect it as a society? Because maybe that's actually also an interesting policy and research question. So, does research come in with a one-year lag? Does it come in with a three-year lag? It's kind of a, an interesting meta science thing.
Seth: Right. And, with changing it at the Bureau of Labor Statistics these days, maybe it'll take even longer to find out what's going on with employment and the economy. But yeah, I... To me, it seems like obvious question, good new data set
Andrey: Yeah, so I think we should introduce the paper and kind of what it does, because I do think that that's kind of what the authors would say. "Yes, but we have this better way, more powerful data set of, that can study this thing."
Seth: Okay. So what do they do? What's the data? In a sentence, they use this amazing data set from ADP, which is one of the world's largest HR companies, they say covering firms with 25 million employees in the United States, and with this really vast data set covering from... Most of their analyses are since 2020. Some of the analyses go back to 2018. They ask what are the patterns we see for people by employment times occupation, and then by the age of the worker. So again, there's a little bit of a conflation here. They want to talk about entry level jobs, but they're actually talking about ages of humans. I don't know if that bothers you, that little- ... Back and forth. But that's what they do. Econometrically, it's pretty straightforward. There's just some straight up, descriptive statistics saying this went up this percent, this went down that percent. There's some event study stuff trying to argue that the, it's all happens kind of right after the start of 2022 with ChatGPT. And then finally, they do some additional controls and additional splits for confounders that we can talk about. You think that's a fair description?
Andrey: Yeah. I think that's a reasonable description.
Seth: Okay. So just to talk about the data for a second, they have to do some cleaning of it. They start throwing out people older than 70, younger than 18. It's not exactly clear why you would need to do that if the data was perfect.
Andrey: They have, they have a balanced panel. They kind of ensure that they're looking at firms that are consistently of a certain size throughout the entire time period they're looking at. And primarily their results are looking at a time period between 2021 and 2025. So firms can certainly change over that time period.
14:14Seth: Definitely. One note I would add here, the small nimble startups are probably using alternatives to ADP as well, or maybe just doing bookkeeping by hand if you're a really micro company. The other thing I would say about this data is there's a lot of heterogeneity in firm characteristics, details on the occupational of the exact occupation. There's some additional text you get from ADP describing the occupations. Lots of stuff on worker characteristics, like where these people live. Doesn't show up in this paper, but I'm aware from this data set there's a lot of different directions you could go with this data if they wanted to keep on pushing. Okay. So how do they use this ADP data? They combine it with two other data sources, which listeners of this show have already learned a lot about. The first is from friend of the show, [laughs] number six I think we're up to now, Daniel Rocks, Eliando et al paper that is called GPTs, for, Our GPTs, which develops a measure of the exposure of different occupations to AI.
Exposure just, meaning it would be really useful to have AI systems to speed you up. Here we're using their secondary measure, their beta measure, which takes into account you also have to make complimentary innovations in addition to the AI to do the job. So that's thing one they're gonna use to, in order to split occupations by more or less AI exposed, and the second thing they're gonna use is what we talked about in our Anthropic AI index, episode, which is this data from Anthropic that they've released on what share of communications with their chatbot seem to relate to different jobs. And then they're gonna do one further and try to figure out whether those requests are more kind of automate-y or augment-y. Um-
Andrey: Yeah, which is something I recall you really disliked.
Seth: [laughs] Dude, I don't... Man, and the worst part is when they say automation, automate-y versus augment-y is really important, and then they cite, the citation for that is Eric's Turing Trap paper, [laughs] book or, essay. [laughs] Which also does not engage with this issue. Something that we keep on coming back to is it really is not useful for talking about a technology in isolation as automative versus augmentative, that really you have to understand the entire economic system to understand whether a technology is going to boost or reduce the demand for a certain kind of labor. So, [laughs] I don't know. My eyes roll super hard whenever I see something like that.
Andrey: Yeah. But anyway, it's still a, it's an empirical question whether these things are correlated with-
Seth: Right. So they're different cut, yeah. Whether or not you want to connect them to automating. They're different uses. And do you want to talk briefly about the econometrics? It's pretty straightforward.
Andrey: Yeah. So, we can just go figure by figure. So figure one, the head count over time for software developers and customer service representatives by age group, and they're gonna normalize this, the script statistics so that everyone is at 100 in the middle of 2022. The early career workers, so those between 22 and 20- 25, the trend changes at exactly the point that they picked to start going down.
17:43Seth: [laughs] How convenient. 2022 is when they invented AI, right?
Andrey: I think so, yeah. Well, and then the other series for the most part keep going up, except that 26 to 30-year-olds are kind of flat or maybe a little declining. Yeah, so before we get into the rest of the evidence, I think this is, a tricky thing to... It's always great to plot the data, of course. But we can't learn that much from a simple plot because we kind of don't know The counterfactual. We just generally don't know the counterfactual, but like-
Seth: Right, it's a very short series
Andrey: ... Even. So they're plotting two occupations, right? Lots of things could be going on. There could be changes in demographics. There could be changes in how long it takes people to graduate college because of a particular pandemic. There could be changes in demand for labor in other occupations that might draw people away from software and customer service.
Seth: Right, maybe this is demand for my-
Andrey: Yeah, so there really is a lot of stuff that could be going on. And, thinking really hard about how to normalize it's just important to do. So I think it's a good figure one, but we need the rest of the evidence. And I do wanna reiterate that to me, at least 2022 is definitely not where I expect this to start.
Seth: It seems like the special time, especially because there's no, acceleration. I'm looking at the trend. So if you wanted to tell a story where maybe '22 is the beginning, but then things start accelerating, it doesn't really look like that. It looks kind of straight line decline starting in late 2022.
Andrey: Yeah, and for software developers, you see it flatten out even before 2022. There's no way this is AI. Just time-wise, like when you think about really capable versions of ChatGPT coming out, this is not the right timeline to be thinking about.
Seth: And I guess the, what I would say here is it's just such a short time series that, I don't know, if we had 70 years, would we see, the different age groups diverge and then come back together and then diverge again? I don't know. I, yeah, right, it comes back to what your prior is. If your prior is everything should... All the different age groups should follow each other perfectly, your prior is I would wanna see the historical time series for how often that happens, then this one event is only one data point, so it can't really convince you.
Andrey: Yeah. I do... I think I wanna bring up something else here, which, you think about, something like customer service. I don't know, how big of a sector of employment that is in the United States, but, if I was a young person, I'd be pretty stupid to enter this field. No offense- ... To people who do it, right? Because even if there isn't, a decreased demand for customer service, it's kind of the first thing we expect to go, if we were to- ... Like, go down the list of things that we expect ChatGPT to be doing. And I guess, Eric and company's other paper has shown that actually customer service become more efficient as a result of AI. So that's kind of a natural thing to think about. So even the question of whether this is a supply or demand story is not answered by a graph like this.
21:12Seth: Well, of course it's got a quantity, right? And, you know.
Andrey: But the normative implications are quite different, right?
Seth: Right. I guess what I'm-
Andrey: If there was a bunch of these people who wanted to become customer service agents, and then they just couldn't, it's very different than people are just looking for, to make a little money when they're graduating, and before they might have joined customer service, but now they're like, "Well, this is a complete dead end and I'm not even gonna, I'm not gonna do this stuff, and AI can do it."
Seth: Right. I can't imagine all the people who are going into customer service saying, "This is not a dead end."
Andrey: Well- ... There are a lot of older folks that work in customer service, as well. It's a job that has been open to remote work for a long time, and there are lots of-
Seth: That's, yeah, properties of the job.
Andrey: You can do it part-time. Yeah, well, I'm just saying is that lots of things could be going on here.
Seth: I guess just to speak to the could this be supply side driven results, we've talked about how there have been predictions for a decade now, I remember talking with you in the Sloan Building at MIT, must have been 2017, about, truck drivers getting automated. And there seems- ... To have been a kind of anticipatory effect of people not developing truck driving expertise before they actually got automated. [laughs]
Andrey: Yes, exactly. Let's continue to the next- ... Piece of evidence. So then they kind of-
Seth: Figure two's just more occupations
Andrey: ... Go a more occupations.
Seth: Yeah, they're just showing that something like home health aides, there's actually disproportionate growth in young youth employment, whereas for customer service, there is, disproportionate shrinking for youth employment in customer service. I guess what I would say here is, if it's this draw away story, you could see a story where, we're getting more polarized into physical versus non-physical tasks, right? And maybe AI, there's some sense in which technology is moving young people into being health aides, but it's not so much that their current job is getting automated, it's that, the wage for being a young, strong person is going up.
Andrey: Yeah. They know that healthcare is gonna be an occupation for a long time and they're just gonna do that.
Seth: Yeah, and society's getting older and more infirm. So there's a lot of anticipatory possible effects here and pushes and pulls. Um-
Andrey: Yeah. And then one of the other things to note is that you might think that of all the time series of all the age groups, youth are the one that are gonna be the most volatile. Because of just how labor markets work, right? It's-
Seth: And we're coming off of the rece- [laughs] ... Off of the pandemic, which, you know- ... [laughs] screwed up a generation of students.
Andrey: So let's talk... We're talking about the confounders, so just-
Seth: We're getting confounders. Here we go. [laughs]
Andrey: Let's just do it. Let's just do it. I mean... Yeah, so I mean, there are kind of two things that one would, off the bat think about here. One is there's a sense that the software industry, driven by low interest rates, completely overhired. For a long period of time. And then a lot of the big tech just started firing people. And we know, like factually- ... Why that happened, and it wasn't AI. It, like-
24:30Seth: Right. It was interest rates.
Andrey: It was interest rates, it was fiscal discipline. There's just a sense that like we gotta cut the fat, you know? Some people aren't contributing, that sort of stuff.
Seth: And almost by the fat is going to be a person who's been there for less time, right?
Andrey: Yeah, or just hiring less is the easiest way. Let things churn. There were a bunch of hiring freezes.
Seth: All right. Okay, now let's, okay. Let me, let me get you back on this explain. One theory here is what we're seeing is not AI is taking youth people, youth jobs. What we're seeing is interest rate goes up, a youth job in an AI exposed occupation is the kind of job where you need to be trained for a year, two years, three years. And so interest rates go up, hiring someone to train them for three years becomes less attractive, right? Okay. So you would say, okay, so then that job loss wasn't caused by AI. But what if the increase in interest rates was caused by AI? [laughs]
Andrey: Come on.
Seth: I know it's too early.
Andrey: It's too It seems too early. Yeah. I'm and just so it's clear, like I think we're just listing other things that could contribute to this trend. I think I'm with you that, certainly AI, the AI story could be there. It could be there especially in the later part of the sample where I believe the timing makes more sense. But yeah, the other really natural one is, man, education, college education, high school education during the pandemic was a mess. It was really bad.
Seth: I was there trying.
Andrey: Yeah, we tried, but I mean, it was not a good one. And I think this, a lot of things happened with Zoom. You hear stories about people aren't like as confident being social. The learning that occurred was much lower. That's been shown in a variety of settings, right? Just taking a class on Zoom reduces learning. You're gonna go into an interview for a technical subject like programming, you're gonna do worse.
Seth: Well, yeah, right. Because like that, that story it seems like I'd suffer more for the social job or the job that isn't gonna get AI exposed.
Andrey: No, I mean, I think to some extent you need technical skill. I guess interviewing is a combination of social and technical. And I'll say this in my career as I'm a terrible interviewer- ... Is that, it's not just about technical, but it's both. And so if you're both worse socially and worse technically- ... You might be less likely to get that job via interviewing.
Seth: To explain this story, you have to tell a story why that affects you more in the AI exposed jobs.
Andrey: Sure. I understand that. But I guess I'm just saying that learning rates in interview processes can interact in interesting ways, and they might interact differently for different types of jobs.
Seth: Fair enough, and it's plausible that you would need more training.
27:32Andrey: I'm not, yeah. Look, I think, I think one thing that we need to think, like always acknowledge in terms of like a macroeconomy is like humility about jobs. Like I don't know the scope of all the jobs in the world, right? And I don't know their interviewer policies. And I, all I know is that for like software engineering jobs, there are like these very structured knowledge-based interview processes that are kind of unusual relative to other jobs. And, other places which have similar structure might be like hedge fund interviews, maybe banking interviews, right?
Seth: Those are AI exposed, though.
Andrey: Yeah. So that just goes to show you like it's just hard to like, it's hard to... This is a plausible confounder, but you know, we need to think harder about it.
Seth: Yeah. And I know that home health aides certainly don't have long technical interviews, so.
Andrey: They're absolutely not. They're, it's a very easy interview process relatively so.
Seth: I just, I, so I just did it with my... We just hired a nanny for my baby, and it really was like I just watched them for half an hour to see if they killed him. [laughs] And they didn't, so they were hired. [laughs]
Andrey: Okay, so now we go to figure three. The nice thing about this paper is it is very clear about what's going on, and it's just we're looking at figures and interpreting them. So figure three uses L&D and all, so GPT, their GPTs paper as a measure of exposure to AI. By the way, what the authors will say is that it's definitely not a measure of automation. Absolutely not.
Seth: It's definitely not this is gonna take your job. It just- [laughs]
Andrey: Absolutely not.
Seth: Two episodes on this, or three.
Andrey: Yes. What we can look at is kind of the dispersion in these trends across these categories. And we see that the darker lines here, the more exposed lines, they flatten out for early career people, but then for other folks, they move lockstep with most of the other jobs. Although there are other lines that substantially deviate from the trend in this plot.
Seth: Yeah, across. [laughs]
Andrey: For example, for 31 to 34-year-olds, less exposed, the two less exposed jobs really increase in headcount. And so-
Seth: And it's interesting, there seem to be a lot of seasonality in the least exposed jobs, right? They seem to bump. So I'd like-
Andrey: They-
Seth: What's the least exposed job? It's like harvesting fruit. [laughs]
Andrey: Yeah. So yeah, there's tourism I'm sure is somewhere in there. You know? Yeah, and these, even in the pre-p- the, one of the interesting asymmetries about these plots is there's one year of pre-period, or a little more than a year of pre-period, and a few years of post-period. So it just looks that in the before period there's less variation, but actually- It's hard to make that visual comparison
Seth: Do you wanna skip four and go right to five, or do you wanna talk about four for a second?
30:34Andrey: Let's talk about five. Five is, instead of, plotting, time series, they just show the overall change in employment by exposure group, and here they pool one to three and four to five. I would just say that I like to keep pooling persistent, especially since-
Seth: [laughs] It does look darn nefarious to do that
Andrey: ... You are slicing and dicing a little bit here. I don't think they did anything nefarious, but just, you know.
Seth: I'm sure it works the same- ... If it's one to two as if it's four to five.
Andrey: I'm sure it looks the same, but it just, you wanna keep that similar, and 30 months-
Seth: But yeah, the re-
Andrey: ... Standard errors would be much appreciated as well
Seth: Yeah, so but Andrey, I'll say I like this figure in some ways more than the plots, the event studies. Because it seems like at the end of the day, you just wanna look at the end and look at the difference between the different groups, and this figure is kind of summarizing that.
Andrey: Well- ... I think one of the reasons I mean, I'm really glad we saw the previous figures- ... Because we see that the timing truly does not line up with AI. [laughs] But here you might think, you don't know at what time the actual inflection point happened.
Seth: So this figure is how I want the results reported numerically. But yeah, of course. The event study gives you timing information. In terms of magnitudes, I like this figure. And-
Andrey: Yeah, so y-axis, just the growth and number of jobs by age group, and we see kind of everything is up except for the exposed jobs for 22 to 25-year-olds.
Seth: I'm sure 'cause they've got 20 million observations.
Andrey: Yeah. Well, they're significantly different from each other. I guess what I would've loved to see is just some comparison to historical fluctuations- ... Over other time periods.
Seth: And it's where I come back to, I don't know about a lot of age times occupation-based research. I'm sure you could construct something using the census, but I haven't seen-
Andrey: So-
Seth: Yeah, I haven't seen this before.
Andrey: So I think there's a literature about how bad it is to graduate in a recession. And that is all about the youth, and all about how much youth employment fluctuates, and how if you're unlucky and you do graduate in a recession, your career trajectory is worse.
Seth: And you've seen that, occupations?
Andrey: I haven't seen that time stamped with occupation. But I think, to me, I'm using that as kind of some evidence that there is more fluctuation at that for graduates essentially.
Seth: Right. Yeah, I guess the meta point here is it's not something special about the skills of young people that's important here, but rather just whenever there's a reorganization in society, the reallocation is gonna happen from young people first. They're more mobile geographically. They're more willing to quit a job and look for a new one than someone who's 50 plus and been in a job forever. So for any technological change, we would expect there to be more reallocation across occupations for young people than old people.
Andrey: Okay. So then fact three is about this automation-
33:34Seth: Wait, we should say the magnitude. Let's say the magnitude. And so-
Andrey: Yeah, so the magnitude is a decrease in employment, for quintiles four and five, so for those most exposed, of about a little more than 5%, and then there's an increase for one to three-
Seth: About 9%.
Andrey: Sorry, for one to three, for quintiles one to three, so the least exposed, about 9%. So, one way to read this is, well, the causal effect here is, essentially around 9 plus 5, so 14, right? So 14% decrease in employment, and then-
Seth: So larger, ooh, larger than the effect that Andrey thought was reasonable.
Andrey: Well, I said order of magnitude.
Seth: [laughs] You were up to 100%.
Andrey: But I guess what I would say here is that, the extent that people are shifting from four to five to one to three, the quintiles, that is, of exposure, right? It has a very different interpretation than if those people... What really we'd wanna see is kind of if we look at people who ex ante we predict would end up in occupational categories that are most exposed. And then what are their unemployment rates and what are their... I think more smoking gun evidence would be if they had high unemployment rates for CS majors and, we have been seeing some news about this. I think would be, in some sense, very-
Seth: Corroborating
Andrey: ... Complimentary. Because if there's substitution between one type of job to another type of job, that's a... Is that a demand or supply side phenomenon? Is a very, [laughs] very live issue.
Seth: Dude, no matter what, there's gonna be substitution. That's the thing, is, nobody's actually- ... Thinking we're gonna, eliminate all youth jobs. So it's always going to be a recomposition. Okay. So this is a simple paper, so I don't wanna overcomplicate it. Maybe we can just briefly say the couple of things they try to do to deal with confounders, the different robustness checks they do. Andrey, do you find this robustness check where they control for a firm times time total employment effect, did that, does that help you at all in terms of thinking that this is caused by AI or maybe one of their other controls or splits?
Andrey: They do also, split things out by augmentation versus-
Seth: Yeah, but we already said that's silly-
Andrey: ... Automation. Yeah
Seth: ... So I'm gonna skip that. [laughs]
Andrey: Well, I mean, I mean, they, and it does, corroborate their story. But I agree that since we think it's silly, maybe that's not the right way to, and not the right place to look at. So the specification with the fixed effects, right? So the idea here is kind of some firms are gonna be hiring a lot, some firms are gonna be hiring a little, and if we think that hiring is more or less uniform across age Then it would be surprising, kind of, like we'd kind of see a deviation from that for the youth that are in occupations that are like so, right? I think it's a reasonable thing to do. I think it's a good robustness check.
36:43Seth: Would you want, for- I assume they tried, because they've got a so much data, I assume they tried a firm times occupation trend. That would, I mean, if they showed me a jump in October 2022 with a firm times occupation trend, maybe I'd be-
Andrey: Well, the key thing for me is I absolutely do not want a jump in October 2022.
Seth: Yeah, well, the-
Andrey: I just think the timing is wrong. The jump of 2022 is what makes me skeptical.
Seth: [laughs] It makes it what it has to be, interest rates, 'cause that's the only thing that fast. [laughs]
Andrey: Yeah. So I guess, yeah, that's... And then I'm like-
Seth: You know what James Carville says about the bond market?
Andrey: What does he say? [laughs]
Seth: He says, "When I die, I want to be reincarnated as either a baseball relief pitcher or as the bond market," 'cause the bond market controls everything.
Andrey: That's true. All right. So I do think, there's something we want to look at in this specification of the fixed effect. They plot it in figure nine, and if we look at the most exposed group, we actually don't see a deviation from trend. Am I reading that correctly? Yeah. In some sense.
Seth: That's much more plausible. [laughs]
Andrey: That's much more plausible to me. But then I think going in with the descriptive stats, where the timing isn't that to me, is a little... Now I'm confused, is maybe what I'm trying to say.
Seth: I think that where we're gonna have to end up is, there's a lot of stuff going on, but maybe AI is part of this, right?
Andrey: Yeah, I think that's fair.
Seth: And then the last thing we should talk about is the wage change. So far we've been talking about employment changes. In terms of wage changes, they see really no effect by early career times AI exposure on wage growth. But I don't know, I'm looking at figure 10 and looking at early career software developers, and it kind of [laughs] seems like it's going up. [laughs]
Andrey: Good time to be a software developer.
Seth: Well, yeah.
Andrey: If you got a job.
Seth: If you... It's a polarizing time, right? That's- ... This, what I'm taking away here. It seems like the story should be polarization. This is the age of giving $100 million to, the super AI expert, right?
Andrey: Yeah. I think what... It's really strange looking at the market for software developers, because one of the things that I'm thinking about a lot is equity compensation.
Seth: Equity compensation. There's also title effects. Or that, when you get paid enough, they stop calling you a software developer.
Andrey: Well, I mean, I doubt that, 22-year-olds are- ... Getting such titles, but more like, all right, so you got a, an equity package, and then the tech industry is booming in the stock market over the past few years. Is this being captured in this data?
Seth: So my understanding-
Andrey: Or are we talking about-
Seth: ... Maybe not, but I'm gonna use just, knowledge of this data set to say that my understanding is that this is taxable income. So to the extent that that equity is getting taxed in that period, you would see it.
Andrey: Yeah. So essentially we're getting, a partial effect. So we're getting, the amount that you get vested today, but if your stock that you're gonna get in a year from now has gone up a lot, then-
39:50Seth: Yeah, it might not show up in this year's income.
Andrey: But certainly that might explain part of this trend for software developers, where early career ones are getting paid a lot more if they have some equity comp and it goes up a lot. But then we've... Well, maybe we'd expect to see that everywhere, so then maybe that's, it's unusual that it's for early career folks that's going up.
Seth: I think the story here, the story for software is polarization, right? That's what it... So we've run through all of the main experiments in this. They do a couple other robustness things that I'm gonna gloss over. Are there any other concerns/confounders that you have not already brought up, Andrey, that you want to bring in here?
Andrey: Hmm. Yeah, so let's talk about one other thing I think it's important to talk about, because it does remind me where this is all going, is kind of a literature that seems very similar to the China shock literature.
Seth: Oh, right. On this one thing, the only look at the downsides of this one thing. And we'll end up with a generation of policymakers who are like, "Did you hear AI took that one customer service rep's job?" "We gotta shut this down."
Andrey: Yes. So just, to recap for those who haven't been following.
Seth: Ator ruined America.
Andrey: It-
Seth: Sorry, that was too strong. If you're listening, Ator, I don't, only mean that 10%.
Andrey: All right. So pretty much there were a lot of people who were trying to understand what trade with China was doing to employment in the US, and also how it was regionally and occupationally affected. And so they came up with exposure measures. Does that remind you of something?
Seth: Ooh.
Andrey: Yeah. And then they kind of found some exposure measure that correlated with some employment losses in a variety of parts of the country. If you read the papers, they're careful about, caveating that, well, just because there are fewer jobs of one type doesn't mean that things are bad because of China trade, for example. Maybe there are other things that are going on that might be good, and so we have to weigh the pros and the cons. But really when that literature was interpreted was just how China trade was bad. And I think-
Seth: When it's literally measuring a difference, right? With the exposure.
Andrey: Yeah. But to really understand the true effects, you need to think about how all this stuff is getting reallocated. And in particular, with the China shock, this was kind of like, well, US firms are getting cheaper inputs, customers are getting cheaper goods. Maybe employment is substituting. I think one of the key things with China shock specifically was, regionally concentrated. And so for the people who stayed in certain regions, it's certainly hard to argue that it wasn't bad, but you know, a lot of people did move and maybe people in other regions benefited, right? So we kind of have to take that as a whole, and I You know, it's not crazy to imagine, and once again, not the fault of the authors of this paper or any other papers, that this type of work, the way that it's framed is we're looking for this bad of AI, and are we really, are we putting in the same effort for the good of AI?
43:10Seth: Yeah, like here-
Andrey: And what would it mean to-
Seth: Here's the thing. Here's the thing. May I- ... May I, may I, to be even snarkier. It seems like you could write exactly the same paper, exactly the same analyses, but the title of the paper is Youth Employment Booming in Non-AI Exposed Industries, right? It's 'cause it's all just the difference, so you could just write a happy story about [laughs] them getting hired.
Andrey: I don't think it's all just... The, I guess, I do think, and you know, we talked about all the confounders so I won't go through them again. But I do think if you have a theory of AI exposure as a substitute for young labor in certain areas, and then you see the labor decreasing in those exact areas, I do think it's more than just, "Oh, we observe a difference," right? So, let's be a little more fair. If you have an interest-
Seth: Except in so far equilibrium, right? Difference the prices come down of these kinds of goods, the prices go bombles up in the other types of goods. So yes and no, right? The- It's creative destruction. Some lose so that we can change.
Andrey: I'd be very worried if overall youth unemployment was going up a lot. That to me, regardless of why, it's a problem. And if AI is the problem, then that may call for a different type of solution than if it's economic uncertainty, right?
Seth: Very local.
Andrey: Why do we care about what is the reason for this? Why don't we just look at trends as well? Different reasons that we address them in different ways. There's policy ways to address things. There's also kind of just like advice. What advice do we give to young people who are entering the labor market, you know? And if the story is, "Well, software engineering is dead, don't go into it", we should be careful about that story versus other stories. I remember when I was graduating high school in 2004, my high school teachers told me that studying programming was a bad idea because, after the.com bust, there were too many programmers.
Seth: [laughs] I remember in high school them saying, "Oh, computers, they're so good at like adding. You shouldn't study math. You should study art because computers can't draw." Yeah, I mean, it's, and yeah.
Andrey: But-
Seth: That's why this matters. [laughs]
Andrey: Yeah. So do you wanna kind of go into posterior mode?
Seth: I feel exactly-
Andrey: Is there anything else you wanna talk-
Seth: ... Ready to go into posterior mode. So Andrey, ultimately, is this a canary or a red herring? This question was the descriptive question. I kinda mixed in some causal statements.
Andrey: Look, I just like that they put the data out there. This, and, getting the agreement to get this very rich data is really impressive. And, it's a plausible story. I guess, yeah, I go back to it's a plausible story that, AI could be involved here. But I think that the timing doesn't line up with when I'd expect AI to start biting, at least in the non diff in diff results. And it's generally, there's so many other things that could be going on that I wouldn't view this piece in isolation as a canary in the coal mine.
46:29Seth: It's not a red herring. So what animal is it?
Andrey: It's not a... Yeah. It's neither, I would say. I think it's, I think it's in the middle. I think we need to, look at this in a variety of ways. I think, to me, a kind of natural place to look at is the employment rates of people who we expect to be going into these occupations based on their majors. Also, major choice is a very interesting question, right? So, if we... This is a supply and demand story. Always hard to tease out, of course. And that would... But you know, it's a job posting, so there's a recent paper that was, released, I think, like yesterday or the day before that was looking at LinkedIn posts. That kind of seems to suggest corroborating evidence on these sorts of occupations.
Seth: The LinkedIn... Yeah.
Andrey: So.
Seth: Maybe we'll do an episode- ... On the LinkedIn data.
Andrey: But I guess what I, you know... Yeah, that paper uses quite a different, strategy to study these questions. Maybe-
Seth: It's a little bit more at the firm level
Andrey: ... More zoomed in on the AI part of things. So look, I think, it's great that people are working on this. I'd expect to have, 100 papers, and we'll see if they're kind of helping us draw out the shape of the elephant, that it is AI versus something else.
Seth: Well put, Andrey.
Andrey: Yeah, what do you think?
Seth: Yeah, so I'll say on the descriptive question, this data set's so good, I would say I'm at kind of 95%. I'm convinced that basically descriptively what they describe for their 25 million people in their firms would well describe the economy as a whole. We talked a little bit about some selection in the data set, but it doesn't seem devastating. In the terms of the causal question, how does this move me in terms of thinking about, is AI particularly bad for people who are just entering the labor force? Would you prefer, is this a net negative for you in terms of share of income, right? 'Cause you might imagine that the share of labor income going to the youth might go down even if the total labor income of society goes up. Just 'cause GDP goes up, even if labor share of income goes down, could still be a good world. But I am moved in the direction of it's gonna be a good world, but challenging for young people.
I think I'm now at maybe 75% chance I'm up to that early stage workers are gonna be particularly challenged by AI. And what's the reason? Well, it's kind of the reason that technology is often skill biased. It's, you automate the simplest thing first, and you have to put that much effort into... And so young people are either gonna have to be in college longer, or they're gonna have to accept essentially trainee roles for longer, where they get paid like they're true trainees and not like they're people who are, doing some useful work and getting training on the side. So yeah, this does dim my vision for Gen Alpha. Of course, they'll make it all up on interest rates in the stock market, so I'm not too worried about them.
49:35Andrey: Makes sense. Yeah, it's funny that you bring up the education thing, 'cause I think part of what's going on in this plot is that people are taking longer to graduate college. Which would mechanic, in certain majors, right? So-
Seth: But, I mean, that's but that's consistent with aggregate educated.
Andrey: Yeah. No, but to me, it's consistent with COVID really screwing things up for- ... Educated people, right? This is kind of the tricky part of all this. I think I have a very strong belief that it's gonna be a weird time. And that makes me worried the most for the youth and the old.
Seth: Weird's bad for the old.
Andrey: Well, weird in the sense that if you've tried to go on your preordained career path- ... The few that we know about, and then all of a sudden that no longer looks the same.
Seth: Oh, so it's bad for type A try-hards.
Andrey: Yeah, it's bad for, those who are just expecting a certain career trajectory. It might be good for those who are nimble, because I think every person with AI is gonna have a lot more capability as a result of that.
Seth: Right. Be a lot more-
Andrey: And-
Seth: ... Last mile problems that show up.
Andrey: Yeah. But someone being a lawyer, right, you start worrying, well, what does an entry-level lawyer do that AI can't do? And they're, I know they're thinking about it, and people are thinking about it before they're even going to law school, right? So just this inherent uncertainty being really amplified right now compared to any time that I remember in the past. And then I'd be kind of worried about old people, to the extent that, if our society is not providing for them as much as-
Seth: Social Security trust fund runs out unless the... Well.
Andrey: And well, yeah.
Seth: I think that's a good place to... I've got this lovely sunset that I'll leave our video users of the podcast with. And we're welcoming, Jacob Avram Benzel to the world. And, I, my prediction, Andrey, is even another 20 years from now, there's still gonna be jobs.
Andrey: I'm with you. I believe in the labor market.