LSE Professor Sarah Bana on How AI is Changing Hiring, Academic Publishing, and Religion
Justified Posteriors Talks Algorithmic Monoculture and Balloon Popping Games
Andrey: One version of the future is a corporate panopticon, where everything we do at work is observed by AIs and measured.
Sarah: One of the other things that’s going to differentiate certain jobs from others is how much freedom you have.
Seth: Are you above or below the panopticon?
Sarah: What’s happening in the hiring space is a little bit different because it’s increasingly mediated by third-party vendors. They claimed that this was unbiased. We found that wasn’t the case.
Seth: Would your long-run bet be on match quality and jobs improving in, five, 10 years because of AI?
Sarah: I’m optimistic that we will come up with different solutions than the ones that we have now to be able to connect workers to jobs.
Andrey: In the Coasean singularity, the AIs will match all of us together efficiently. [laughs]
Seth: Welcome to “Justified Posteriors.” Today, we’re pleased to be hosting our colleague, Sarah Bana, of the London School of Economics and Stanford Digital Economy Lab. We’ll be talking about AI and reviewing, AI and Islam, but first, AI and cultural homogeneity. What is cultural homogeneity, and do I have too much of it? I’m Seth Benzell, coming to you from Chapman University in sunny Southern California.
Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. I’m very curious to learn about the monoculture of podcast guests that exist [all laugh] in the world. Welcome to the show, Sarah. I’m very excited to have you.
Sarah: Thanks so much. Longtime listener, first-time caller.
Seth: Yeah, it’s so great to have an ascended fan on. Can we briefly ask you what’s your favorite episode before we get into it?
Sarah: Oh, okay. That’s a great question. I did like the two-parter that Bo did. I thought that was really fun. I work in a similar space, and so I’ve been thinking a lot about AI and signaling, and so then, that was definitely a very fun episode. And I think it also got a little bit into metaphysical things, as sometimes you do.
AI, signaling, and algorithmic monocultures in hiring 2:06
Andrey: Before we get into specific papers, do you think that AI is already ruining signaling? So the idea here would be that it’s so easy to apply to jobs that this, traditional way of applying no longer works. And I guess the flip side, the counterargument maybe is that even before AI, it’s been very easy to apply to lots of jobs. So, how would you think about measuring whether this is a big deal in the economy?
Sarah: Yeah, this is a great question. I think we all saw Jesse Silbert present at Summer Institute, and he’s got this paper that looks at the correlation of resumes, and cover letters from their text. And he finds, with a co-author, that almost everybody is editing their cover letter to match the resume text now. And that makes sense because it’s a lot cheaper to do those things.
Seth: That’s like one of the things you coach people on, right? People have finally learned that lesson.
Sarah: Exactly, except not everybody was doing it before. And so if not everybody was doing it before, then you had some signal, at least in terms of interest, in terms of people’s applications. Right now, it’s really challenging to differentiate effort, and interest across the board. I think that on LinkedIn, I saw one of these tools where you can upload your resume, and it allows you to swipe through jobs and automatically sends your application custom to that particular job. [laughs] So I think-
Seth: AI custom.
Sarah: AI custom. And you can do this with, pretty much any LLM right now. And so in that case, that very first step just means that people are applying to a ton more jobs because the manual cost has just decreased substantially.
Andrey: And now I’m thinking, I don’t think we have a good measure of this in the literature. As like, what is the causal effect of a customized cover letter? Because, if I’m an employer, I don’t have that much experience for non-academic jobs. Well, even for academic jobs, I know customized cover letters kind of often don’t matter. And for non-academic jobs, other things seem to matter a lot more, like references, but also about your objective qualifications. If you’re an entry-level, college worker, for example, your GPA, the college you went to, your major. If you’re an experienced worker, where you were working previously, what your role was. Those seem to be so much more important than a customized cover letter. So, while I agree in principle that there is some signaling value there, is it first order? Do you have any thoughts on that?
Sarah: It’s a great question. I don’t think there’s a first-order, response there. I do think that there’s a lot going on here. The other thing that’s happening is people are updating or writing their resumes differently. And so whether or not you worked as a barista and now you’re transitioning to an office job, you would mention that you worked in customer service before, and of course, that means that you’re good with working with customers. Now, how much you would tailor that to your job now, I think, is the other element that’s changing. So, you’re right that a lot of jobs don’t necessarily have cover letters. I think it differs a lot by industry. It seems to differ a little bit by country too. And you can also adjust your resume. And maybe the more important thing is that it’s difficult to verify whether or not people actually have these skills that they’re purporting in their resume. And so, that signaling value goes out the door as well.
Seth: Right. And maybe, if I can jump in here, so like maybe to make a technical observation about, in my job, I did this with Python. You need to know epsilon about the task in order to write a plausible CV line, right? And that you can automate now.
Sarah: Right. And we know that LLMs in other contexts have a tendency to overreach. So for example, they’ll make causal statements when they’re not, when there’s no causal statement in the paper. And so then if that’s happening from a resume perspective, then I think employers are having a lot of trouble, figuring out who is actually a good candidate for their role.
Seth: All right. And on net, right? ‘Cause there’s this also this sense that maybe AI boosts the already able to communicate their goodness more than it boosts the bad. So like, so that’s the kind of the idea in Bo’s paper. Yeah, so talk to me about why you think that’s not showing up.
Sarah: I think that is showing up. Bo’s paper shows that it helps the bottom more than it helps the top, right? Then the field is more level. It’s more difficult to differentiate the top performers from the lower performers. And I think one consequence is just more stages in a job process. Now, we don’t have the data on this either, but I’ve heard of people having, let’s say, four or five interviews. That’s not uncommon for us when you have multiple- …When you speak to multiple people throughout the course of an interview day. But in corporate jobs, I don’t think it was as common to have four or five interviews before you got a job.
Seth: So you see substitution into these more human forms of evaluation.
Andrey: You might be right. I’d love to see some data. But I’ve been in tech industry for a long time, and there has always been multiple [laughs] rounds of Like, you have a, you go on site, you have a bunch of interviews in a row. I think that’s pretty standard, for white-collar work. It does, it does raise the question, do we have any data on, are there any good surveys of this sort of stuff?
Sarah: Yeah. So let me answer the first bit, which is before you would go on site for a tech interview, and that would allow you to meet with multiple people. Other people would go on site as well. I think now with remote work, you don’t have to schedule everything on one day. And people can discreetly take an interview, whether it’s they’re working from home or they’re working from the office and, they don’t have a meeting scheduled at that time. And so I think that’s another component that is exacerbating this problem. The problem is, yes, we don’t have good data on this.
Seth: Okay. So we’ve talked a little bit about kind of this general problem of we got AI, people are trying to get jobs, maybe it’s making this matching problem more difficult for whatever reason. You have a paper and are thinking about algorithmic monoculture, especially as regards to hiring. So tell me, what is algorithmic monoculture? Why might it happen in hiring, and why does… That sounds bad. Monoculture sounds bad. Tell me, is it bad? [laughs]
Sarah: When I initially thought about the concept of monoculture or homogenization, I was thinking mostly about, content diversity. So if we think about Spotify, it might recommend The Daily to all of us. But if we actually have a preference for good economics of AI podcasts, then we would seek out Justified Posteriors, let’s say.
Seth: Yes. [laughs] It’s people’s TL;DRs. [laughs]
Sarah: What’s happening in the hiring space is a little bit different because it’s increasingly mediated by third-party vendors. And so it’s not actually the firm that is seeking out people. They are using a tool that’s provided by a third-party vendor, oftentimes an algorithmic tool. And then you’re limited in terms of the inputs that the algorithmic tool is collecting. Pymetrics is a game-based algorithmic hiring vendor. They’ve been bought up by Harver now, and so-
Seth: How big are they? Are they, do people actually use Pymetrics?
Sarah: It was bigger before. It seems to have dropped off a little bit now. I would say maybe a different way to measure it is the types of firms that clients of Pymetrics. So a good example is like Accenture. Another good example is JPMorgan and Chase. And so you can see from Pymetrics’ website that these were all previous clients of Pymetrics.
Andrey: What sort of roles? So like, if I’m applying to be an investment banker at JPMorgan Chase, are they gonna make me play these games?
Sarah: Yeah. So I think that certainly those summer investment banker internships that eventually will lead to investment banker roles, they oftentimes go up to director level roles. But also, they’re using, clients are also using this stuff for warehouse workers. And so the other interesting thing is the same set of games, 12 or 16, are the two configurations of Pymetrics, the same set of games are being played by workers who are applying to these different types of roles.
Seth: Give us some example of some of the games.
Sarah: So one example of the games is this called the Balloon Analogue Risk Task. And what you’re doing is you’re trying to figure out how risk-averse somebody is by how big of a balloon they’re willing to make. A bigger balloon makes more, money.
Seth: Points. [laughs] Wait, do they get money or do they get points?
Sarah: They get some value associated with the balloon, but the balloon has some probability of popping, and presumably that’s increasing in the size of the balloon. And so you can measure somebody’s risk aversion.
Seth: Wait, pause. This is the most interesting thing. Wait. So do I know what the what the probability distribution on the balloon exploding is, or is this more like ambiguity aversion than risk aversion?
Sarah: You don’t know the probability distribution.
Seth: [laughs] So we’re trying to measure people’s ambiguity aversion with this game. And I don’t know, maybe that might be wealth, you point out in one of your blog posts, right? Maybe that’s wealth relative. Should we really be comparing a rich person playing the point game to a poor person playing the point game, right? How do you even think that tells you anything about the person? [laughs]
Sarah: Right. And Pymetrics says that this tells you something about the person. Maybe you want a more risk-averse person for maybe a technician role.
Seth: And we agree to this, right? Did we agree this isn’t even risk-averse? [laughs]
Sarah: You’re, you’re right. You’re right.
Andrey: I’ve, I’ve just a question about just the practicality of the system. It’s used for, an initial screening. At what point does it come into the process? ‘Cause obviously it’s a multi-stage process. And is it binding, right? So, if I’m a hiring manager, do I have to use this prediction?
Sarah: So, I don’t think you have to use a prediction. In practice, what we see is that you’re scored green, yellow, or red based on your score, and that feeds directly into the applicant tracking system that you’re using. And so the top candidates are going to be, green, and the bottom candidates are going to be red, and they usually stop at some point. So you can use this however you want, and it cuts off at 50%. So green and yellow are above 50%. And so this is an initial screening, and the way we’ve seen it used, and of course, there, this is the, one of the challenges of research in this space, is that you don’t get to talk to every single employer about how they’re using it. And it looks like they are just screening out the candidates that don’t get above 50%.
Andrey: Are we thinking about, here’s a list. You have, 100 applicants. By default, 10 only show up on the page, and no one clicks to page two, and this is, and they’re sorted by only this. Or are they sorted by something that takes as an input this game-based signal, but also includes other signals? If I got a referral, would I be, where would I be? Even if I got a referral, I’m not gonna spend any effort on this stupid game. Am I gonna be screened out? [laughs]
Sarah: In practice, I think that most firms are using this as a screening tool, except for cases where they, you might have had a prior internship, with the firm or some other very compelling reason to move forward. And so, I think that the dominant way we see it happening is in this screening out way, and unfortunately, we don’t have better data on exactly how it’s, it’s being used across all these four million applications.
Andrey: Okay. And then here’s the other thing. I remember hearing pitches for this sort of product back in the day, over 10 years ago. Even at some point, a friend who’s a venture capitalist asked me to talk to a founder who was trying to pitch this. And I came away with this, there is no evidence that this is a good way to do screening, right? Previous guest of the show, friend of the show, Bo, has done a lot of work in designing screening systems algorithmically for large firms, and at least that included a randomized controlled trial. And even there, with a pretty sizable randomized controlled trial, it’s very tricky to figure out what’s a good screening tool. And here, it’s just, it seems to me these people are like, “We read some pretty crappy old psychology research that says that-”
Seth: People who click on balloons are bad employees.
Andrey: Yeah, that this is an important thing, and therefore we’re just gonna go with it because it sounds cool. Is there any better evidence that this actually works?
Sarah: I think that it depends on what the counterfactual screening tool is, right? And in a lot of cases, I do think that this is being used partially because it’s very challenging to manually screen through all these different processes. And so, what this process does is it splits your application pool in half. Is it helpful? Potentially. And let’s say the applicant quality of the people who continue are 10% more like the people that you currently have, and then you’ve screened out 50% of your people otherwise.
Andrey: But I could just flip a coin, right?
Sarah: You could. You could. And I don’t think people are comfortable with that. I think that’s a great idea, and I don’t think people are comfortable with that. These are positions that, on average, have over 2,000 applications submitted to them. And so, these are high-volume positions, and somebody’s gotta do the work to screen them.
Andrey: So I think it really depends on the role here, right? Now let’s, let’s talk about the investment banking intern, who I’m still skeptical is truly in this data. But you have much better signals in, GPA, and, even before LLMs, I think, Bo’s work shows how feasible it is to create just a simple algorithm that takes these signals and sorts them. And then my other experience there is, there are actually, conversations between people hiring and, professors, “Who’s your best student?” This sort of stuff that’s happening, right? So there, I truly think this can’t be the counterfactual that you have to flip a coin. There are better signals. Or, warehouse workers, like Uber drivers, we’re talking about, things like, I know Uber is now using that. There, it’s kind of like you have an undifferentiated, you have essentially a, you have no signal at all. And so maybe this is one way to generate signal. And by the way, all it could mean is that this person cared enough to even do the game. Because honestly, if someone told me to take a game, I’m like, “Are you crazy?”
Seth: It would just be a desperation screen, right?
Sarah: Part of the reason why people like these platforms is because more than 95% of people complete these games. And so, it’s not a desperation screen, I don’t think, in that way. But I don’t think that this is precluding doing all the things that you just described, Andrey. I think that these are stackable elements of the process. And it’s just about limiting or taking that funnel, which is just so large, and winnowing it down.
Seth: And even if it’s 1% better than a coin flip, aren’t you willing to pay Pymetrics, $10,000 for that?
Andrey: No. I just like the entire premise of this product that makes no sense to me. I don’t have a strong prior that, people playing games similarly to my current employees is even positive.
Seth: Well, you have a strong prior that everything in the universe is correlated with everything in the universe, right?
Andrey: But I could imagine a negative correlation, Seth. That’s what I’m saying. The other thing about, people completing the game, effort is a part of completing the game. I am completing a bunch of surveys, but I spend very little effort on thinking hard about those, decisions, right? So…
Sarah: Yeah. So one of the other reasons why I think people were eager to use Pymetrics is because they demonstrated that they were less biased than certain alternatives, or that their… They would tweak their models and de-bias them. And so in that sense, that was valuable for, regulatory purposes.
Andrey: So where does the model come in, right? Because we’ve, we’ve, we’ve played a game. We’re gonna pop the balloon. That-
Sarah: Boop.
Andrey: …The information is you’ve popped the balloon or not. What is the model doing? Is it saying, “Are you similar to the employees of this firm in this role or just general employees in the firm in terms of how you pop the balloon?”
Sarah: Yeah. So, they’ll take 50 employees from the firm, and they’ll compare those 50 employees, and they’re supposed to be in the role. And so those 50 employees will be compared to not applicants to the role, but a non-50 employee subset, and they’ll create a classifier, to compare not just the outcome of the game, but all the inputs of the game. So there’s multiple features in this process, a very high-dimensional feature vector.
Andrey: What are those features? Just, I’m, I’m missing that.
Sarah: Time, click, the way you’re clicking or time between clicks.
Andrey: That is crazy. These people are truly stupid. Oh my God. Wow. I just like, I would just fire anyone who purchased this tool.
Seth: So your concern, Andrey, is that there’s some, hugely overfit model in the back of this Pymetrics thing, right?
Andrey: Oh, yeah. This is garbage. This is statistical Like, I could design something better in a day.
Seth: Would it involve a balloon-picking game?
Andrey: Clearly. Yeah.
Seth: [laughs] Andrey, I guess, I guess maybe this will be the last thing we say on the subject. I share your intuition that we cannot learn whether you will be good or bad at your job from this. But is it really so alien that, you know…
Andrey: Yes, because there’s there’s nothing related to outcomes, and the incentives of the employee, how the employees play the game, how much time they spend on it especially. Like, what are their incentives? You know, what else do they have to do that day? You know, the idea that an applicant should spend a similar amount of time as an employee playing this stupid game, why would I expect that to be true?
Seth: Maybe you distinguish between the employees in the company that you think are good and the ones you think are bad.
Andrey: But they haven’t. They haven’t. As far as I can tell, they haven’t. By the way, to have 50 employees in that specific role is already a very specific set of roles, right?
Sarah: I think that the interesting thing here is that, for me at least, they claimed that this was unbiased or met the thresholds associated with, the New York City rule, the four-fifths rule. And it turns out, we found that wasn’t the case. And so for me, there’s another element of vetting the software that you do have, which I think is important as well.
Andrey: It’s always true that hiring has arbitrary components in it, and I, and I think the point that you’re making in this paper is that potentially relying on these systems is creating excess correlation. So, before, the hiring managers of the 10 different firms, they, each liked a different color. They like, red, green, blue in your, in your CV, and if you had that, then you would get the job, and otherwise you wouldn’t. But because they have different preferences, and it, by the way, it’s an open question. In reality, do they have different preferences? And obviously, there has been, work in economics before on how correlated those things are. But now, to the extent that you’re truly just relying on a system, if that system has a bias, and it could be a bias for anything, right? It could be literally bias for people who use Arial instead of Times New Roman in the CV, and that’s creating this kind of adverse, outcome for that, right?
Sarah: So I think that if one firm has a preference for Arial and the other firm has a preference for Times New Roman, then we all should be fine, yes, because, the people who, write their CV in Arial will go to the Arial firm. The people who write their CV in Times New Roman will go to the Times New Roman firm, yes. But if everybody has the same preference, then yes, we end up in a bit of trouble because we’re screening on this arbitrary preference, and because we’re using an algorithm to do it’s being done routinely in a way that wasn’t being done before. I think that part of the promise of digitization is to try and match workers to jobs better.
Andrey: And this is, in the Coasean singularity, the AIs will match all of us together efficiently. [laughs]
Seth: I know that’s kind of one of the themes of the podcast is, who will win the race, the signaling or the screen, the ability to signal with AI versus the ability to cloud the field with AI? Would your long-run bet be on match quality and jobs improving in, five, 10 years because of AI?
Sarah: I think that I’m optimistic that we will come up with different solutions than the ones that we have now to be able to connect workers to jobs. I think that’s when one of the things that we joked about was how relevant is this task to the actual job that you’re doing? And that’s one of the challenges that up until now, we haven’t been able to simulate work well enough to be able to actually put somebody through the wringer in terms of the job that they’re intending to do. We would use their resume, but actually understand, if you could give somebody a test day at the job and watch them and then measure their performance, then you could get a much better sense of who could actually do the job and who couldn’t do the job. And presumably, you could say, “I know that I can teach these traits. These traits are harder to teach or take longer to teach, and so therefore, I’m going to choose a candidate who is, good at these traits, but maybe more teachable or, could put in the effort on these traits, and then we could get a worker like that.” I think right now, a lot of job search is not that.
It’s these arbitrary things applying to, submitting a resume, right? And it’s, it’s kind of an antiquated technology, to be completely honest with you. And so I think that, yeah, I certainly hope that we improve at this because we should be a lot better. Human capital, efficiently using human capital is, one of the most important things that we should be doing as a society. And so I think we could do a lot better.
Measuring work: Surge, Mercor, GDPval, and the corporate panopticon 25:57
Andrey: I’m gonna make a claim.
Seth: Yes, a hot take. Thank you, Andrey.
Andrey: The people who are most seriously doing labor economics in the world are people at Mercor and Surge AI and other Expert data labeling firms, because they’ve actually, in a very precise way, quantified who amongst their potential experts is actually an expert that’s good enough to judge the output of LLM work. And they have a lot of incentives to get this right, and they’re measuring it at a level that’s never been seen. And I, and it’s worth kind of, for example, watching, someone like Edwin Chen, the CEO of Surge, give a talk about this. And it’s like, compared to what they’re doing, Everyone else is not even serious, right? [laughs] Like, I mean, obviously, some jobs are not just about expertise, there are other aspects of things. But we can get to that world. And here’s, here’s kind of a question for you. In your, in your paper, people are playing these games, there’s some sort of similarity score, and then it happens to be that certain groups are disproportionately promoted or not, even though their race or other signals are not part of that algorithm, right? Now, suppose we did the same thing, but instead of playing these very arbitrary games, we’re doing whatever Surge or Mercor were doing. What do you, what would you make of that?
Sarah: So maybe one of the other episodes I liked was the GDPval episode, because I think that the whole idea of GDPval is exceptional, that we should start taking our benchmarks to things that workers are actually doing, and we’re going to spend a tremendous amount of money, but luckily, they have that money to, generate tasks that people actually do, have other people do those tasks, and then have a third group of people evaluate tasks that are being done by AI versus humans. And I think that whole process of generating tasks, is something that is, we talk about tasks all the time in labor economics, and that process is actually turning a task into something very, very clear, and actionable and understandable. So I think in many ways, I’m thankful for AI’s involvement in all this, in labor economics, because I think that we started to learn so much about labor economics because of AI. And I do think that, evaluating some of this performance is really important. Yeah, up until now, we haven’t been able to say, “Okay, well, Seth is really good at this component of his job, and Andrey is very good at this component of his job.”
Andrey: I have expertise in being bald, and Seth has expertise in growing a lot of hair. [all laugh]
Seth: Comparative advantages.
Sarah: And one of the challenges that I sort of stumbled upon right there, but is a big one, is that most work is done in teams. And the truth of the matter is, Seth and Andrey collectively generate this podcast, and without Seth, it wouldn’t be the same, and without Andrey, it wouldn’t be the same. And so what are they individually contributing to this podcast? Now, that’s a, that’s a part of the process where we don’t have as good attribution that I think we kind of need, right? What is it that Andrey brings to the table that meshes with what Seth does that allows you to produce this good podcast? And if we had a guest host, let’s call her Sarah, what would Sarah have to do to be able to, contribute to this production function, such that the conversation went in the direction that we want it to go in, and there were sufficient hot takes, right? That’s a measure that we care about on Justified Posteriors, and so we want to, we want to optimize with respect to that, but maybe there’s some, curvature to that. We can’t have too many hot takes.
Seth: No, lexicographic over hot takes.
Sarah: We can’t have too many hot takes.
Andrey: We need to create as many TikTok viral TikTok clips as possible.
Sarah: [laughs] And so, I think that’s sort of the, very interesting part of the frontier, right? Where I don’t think that these firms are contributing as much because they’re not involved in any sort of collaborative efforts, and most work is collaborative.
Seth: At super task level. I think understanding, job level production functions and organizational level production functions is something I’m really excited for people to dig into with AI tools. Andrey, I want to counter you and suggest that it is not Surge that is at the forefront of human performance measurement, but in fact, my old standby, sports science. Think about sabermetrics. Think about people being tracked on the basketball court to one half of an inch of their arm angle.
Andrey: Yeah. I’m with you. Yeah. I just— Those are very unusual jobs, but I agree that, [laughs] that sports measurement is quite advanced these days.
Seth: There you go.
Andrey: I kind of view one version of the future as a corporate panopticon, where everything we do at work is observed by AIs and measured, and that creates some very interesting dynamics, right? It— In some sense, it’s like the economist’s perfect market, where we’re perfectly incentivized to do the optimal thing.
Seth: Andrey, I’m gonna I’m gonna intentionally do my job wrong so it gets the AI bad data, so the AI does my job wrong. That’s the meta game.
Andrey: The AI is not gonna be fooled by your [laughs] silly games.
Seth: Even if I poke the balloon at the right time?
Andrey: [laughs] but it does also kind of, yeah, raise this question of human preferences for flexibility, for dignity, for autonomy. And of course, we have so many jobs already that are, well-monitored, your call center jobs, various factory jobs, where we kinda, some people take it for granted that those are very well observed, right? But it’s might be coming for all of us.
Sarah: I think that when you talk to blue-collar workers, they’re much more aware of it than we are. Or, an Uber driver, knows his or her braking is being very well observed, and they can receive incentives for braking better and braking worse, and, or being on their phone. And in some ways, that’s very helpful because, it’s a safety concern, and braking at the wrong time or braking too little is associated with crashing. And so, I think in that sense, it’s, it’s good. But yeah, I do think that one of the other things that’s going to differentiate certain jobs from others is how much freedom you have. Nobody right now monitors our time very well. We could be doing whatever we want. In some ways, we are. We’re on a podcast. We don’t have to, explain our involvement in this process as valuable to our research, or maybe some of us do.
Seth: I have to explain that to my wife. [laughs]
Sarah: And so, in that sense, our jobs differ a lot from other people’s jobs in that way too. And I think that’s going to continue to be a characteristic of, quote unquote, “good jobs,” is the amount of freedom that you have to perform it, whether it’s at times of day, or it be involved in certain activities that you don’t have to justify.
Seth: Are you above or below the panopticon?
Andrey: Yeah. One very natural labor economics framework of thinking about this is, you get compensated in a bundle of things that are not just money. And so, as you have better outside options or as you’re wealthier, then you can select into jobs that compensate you in freedom rather than, In pure financial terms.
Seth: If privacy is a normal good, the panopticon vision, it doesn’t happen, right?
Sarah: So, a lot of the AKM work, though, finds that positive amenities are correlated positively with wage, right? And that’s not trying to move away from a compensating differential story, that still could be the case. I do think that it’s really important to remember that there is more free coconut water at places where people are getting paid more.
Andrey: One thing that plays a hand there is incentives, right, as well. So, you need less overt monitoring if the person is a very career-motivated person who wants to Advance and innovate and create into the future, right?
Seth: You have to monitor those guys harder ‘cause those are the cheaters. Those are the people who are so desperate they’ll do something crazy.
Sarah: You can also argue that it’s operating in the other direction, right, Andrey? If you’re— if you have to use your next paycheck to be able to feed yourself, then, to what extent are you focused on creativity in the first place, right? Don’t you just wanna perform the job and then get out and then eat? And so, I think, I agree with you. I agree that monitoring is less valuable for creative, career-oriented people, but I sometimes wonder if we’ve— if it’s a bit of a chicken and the egg problem.
Andrey: I think that’s an interesting point. I think probably for things like factory work or warehouse work, and by the way, I do not speak on behalf of any large corporations in this regard.
Seth: That has warehouses. [laughs]
Andrey: Yes, that has warehouses or robots or anything. [laughs] you know, one way to think about it is it’s a part of it is a time preference story, right? It’s certainly true that oftentimes the— there’s a temptation to slack, especially in physical work. Time preference is highly correlated with career success in a lot of ways. And so, people, might know that they need the job for their next paycheck, but they might still shirk or they might still… you know, do other things that are, that are counterproductive. It is correlated with some of these things.
Sarah: Yes. And I think that there’s, I mean, probably a lot of us are shirking minute to minute or hour to hour.
Andrey: Totally, but that’s a compensating differential. And this is the outside option argument, right? Is it, other than the workers, at the warehouse don’t have that many outside options. They’re kind of- …More or less under-franchised labor. So, they know that if they, quit, the next person will just come along, right?
Sarah: And I think the other thing, and this is a podcast theme too, which is like, not all technology interventions are the same. We’re not just adding AI to hiring, and we’re looking at a very specific tool that’s being used. And I think that others are also trying to look at other tools that are being used. I would argue that it’s going to be more helpful to look across firms because, then you’re not looking at a firm-specific intervention. And so, maybe encouraging that kind of, additional research, which I think is, important.
Seth: Additional research. And then you’ll screen for who gets to do that research based on how they pop the balloon.
AI and peer review: “Fighting Fire with Fire” 37:48
Andrey: So, this brings us to our natural topic. So, they— let’s say these people do this research, and they submit to a journal. Why would they do that? Shouldn’t— can’t they just send it to the AI and the AI will referee it for them?
Seth: Exactly. Just send it to- …Just post it online. We have the AIs ask, “What’s the best paper in SSRN?” And then they’ll rank naturally.
Sarah: Yeah. I don’t think our tenure requirements are written like that. [laughs] And if they were, then I think that would lead to a very different equilibrium. Absolutely.
Seth: But let’s do it. So, let’s think through that thought experiment.
Andrey: You’ve written a paper about this. So, what’s your, perspective?
Seth: AI and reviewing. Save us.
Sarah: So, I think first and foremost, we’ve probably all seen a lot of or increased submissions. And I think that makes sense because AI is a productivity tool, and it increases productivity. And so, if the number of papers that have been submitted to both journals and conferences has gone up on the order of 20% in certain cases. And so, the question is, now what? Now what do we do with these increased number of submissions? And AI appears to be quite good at least identifying a certain set of problems in academic papers. And so, we developed a workflow where you have an AI review as a first-line reviewer, in a journal process. The interesting thing here is that after you get the AI review as an author, you get the opportunity to respond to that AI review. So, you can say, “Well, I hadn’t initially run this as a staggered diff-in-diff, but if I do, then these are the estimates on the beta and the standard errors. And so, my results are quantitatively or qualitatively similar.” or alternatively, you could say, “Yeah, of course, this is only true over this particular range, but that’s the relevant range.” And so, you can respond to the AI review.
Seth: Is the idea I should change my paper, or is the idea that I have a cover letter, or either choice?
Sarah: Yeah. It’s, it’s, more like the cover letter where you get a you submit a supplemental document. And that supplemental document, along with the AI review and the paper, will go to a traditional human reviewer. Now, the human reviewer, therefore, has hypothetically a little bit less to do. The AI review has read the paper and commented on a certain subset of concerns. The authors have potentially responded to some of these concerns. And so, they can evaluate the paper hypothetically much faster in this case. That’s the plus side. Potentially, there is this correlation between the AI review that gets induced into the human referee or reviewer’s signal of the paper, and that could very well, deteriorate the process because we’re all looking at the same input.
Seth: I’m substituting my own expertise for the… I get the information, it’s trustworthy enough, I don’t add my own view to it.
Sarah: Yeah. Or just the process of conditioning on the AI review. You could be adding less new information here. And so, these are the trade-offs, involved in this, AI-infused workflow that we’ve, written up for Management Science, and we are starting to experiment with for a conference. The idea here is that we don’t know the optimal workflow. Should we give AI to both reviewers? Should we give AI to one reviewer? Should we give AI to no reviewers?
Andrey: I think there’s a question. Not all AI is created equal, obviously. Now, I personally think that it’s reviewer malpractice not to put a paper through some AI tool in the review process.
Sarah: That’s a hot take, don’t you think?
Andrey: Oh, yeah. [laughs] But, the others are wrong, so I don’t know what to say. [all laugh] I mean, look, the AI is way better at me, than me at, spotting inconsistencies in papers. And yeah, I should obviously mention, I’m an advisor at Refine, Inc., a leading purveyor of such services.
Seth: He pays me $50 every time he says that. [all laugh]
Andrey: But, Refine is a pretty, heavy-duty one. Even just throwing it into a simpler tool has a lot of value. You can talk to the paper. You can ask it specific questions. And that kind of raises the question of what does a reviewer- … Doing.
Seth: They provide taste.
Andrey: Whatever taste is, right? And so there’s a, there’s a way to potentially… Well, one is like AI could eventually, have its own taste, how, that we might value. But you can design a review system that focuses on Concerns, atomic concerns, without making a judgment about, well, is this a big enough contribution for the QJE? Only Larry Katz can make that decision.
Seth: Right. So it’s also- …Distinguishing between a technical review and a judgment or a-
Andrey: I think that’s probably pretty good, but I think the high-level downside is maybe that referees take a little bit less effort thinking about the paper. I think that might be, the biggest potential downside. The monoculture, I’m a little less concerned by, but because, yeah, it identified that this number in your text is not actually corresponding to the number in the table, or that, you forgot to cluster your standard errors. And I just… To me, it seems like very second order, yeah.
Seth: No, I get, I get that, Andrey. But again, it’s a black box. For all we know, it’s more likely to catch the technical error in style one of paper than style two of paper, right?
Andrey: It’s so third, that’s so third order, Seth. But also-
Seth: Fair enough
Andrey: …This is where, this is where, evals are really important, right? It’s a much more… In fact, we’re much more confident about that than we are about three random people half-reading a paper that’s submitted to Management Science. I mean…
Sarah: Yeah. So I think that your concern, this, which we call shadow AI, right? So people putting papers into AI tools as part of the referee process. I think that’s another component of it, which is, okay, well, if we’re, everybody’s gonna do this, then why don’t we make it clear what the prompt is, what tool is being used, and share that, ubiquitously across a review team? And that way, it’s very clear to the reviewer, the human reviewer, exactly what their job is, which is, it’s not to find these, nitpicky mistakes, and/or they should be responsible for evaluating the contribution. And so, I think that these two things are related. I think that a lot of people would be very uncomfortable with you, Andrey, saying that you’ve put their paper into one of these tools, because people are concerned about intellectual property concerns.
Seth: The AI reads GitHub.
Andrey: They’re all working papers in our field. The concern is greater for other things. You can, also submit them to AI tools that have terms and of condition that don’t train on the data. I feel like this is a BS concern. This is cope.
Sarah: Yeah. I’m, I’m good with the using these AI tools that have terms and conditions that don’t train on your data. And, but in that case, shouldn’t the journal be responsible for this process, as opposed to it being done at the individual level?
Andrey: Well, you’re concerned about correlation.
Seth: It’d be less correlated-
Andrey: It’d be less common
Seth: …If we all did it our own bullshit ways.
Andrey: I agree with you that to the extent that there’s compute, and to the extent that this is so cheap, it should just be done, it’s not crazy to think that it should be done ex ante, right? Because at least some version of it, by the way. And by the way, the equilibrium is very obvious. If you’re an author, you’re, you’d be crazy not to put this into an AI tool yourself, because you know those mistakes are gonna be caught or those issues are gonna be caught, so you might as well, think through them ahead of time. I think the bigger question, rather than how do we reform our review process, is the format of the paper in the first place, right? ‘Cause the way in which the paper is currently written Is kind of a smorgasbord, especially in our field. Part of the paper, and you see some people like John Cochrane are just masters of this, make it really fun and interesting to read, right?
And that’s an important skill of the paper. But then part of the paper is written for your three referees, and no one else cares at all about this stuff. And right now, it’s all blobbed together, which makes for a poor reading experience. There’s no reason to think that we just have something customized for me and for Seth and for you that the AI does for our objectives when reading the paper. And therefore, one paper artifact, maybe it’s like the version that should exist is maybe like the intro, like the very compelling auth-authorial voice should be in there somewhere. But for a lot of parts of it, we don’t need that. We just need to talk to the AI about like, “Explain to me what’s in this table.” Like, “Did the, did the authors do anything about this particular concern?” You know, right? You’d rather be talking to the paper.
Sarah: Yeah. I think that for me, I would like to play with the data of the paper, and that’s one format that I think has been recommended too, which is like, why aren’t people’s papers Jupyter Notebooks?
Andrey: Well, Jupyter is a terrible format, but it’s, that’s a separate point.
Seth: [laughs] Wait, let’s hear that take. Let’s hear that take. Andrey, let’s see, why does Jupyter suck?
Andrey: It doesn’t work well with version control.
Seth: It’s terrible.
Andrey: The ordering, the ordering of the cells doesn’t make sense.
Seth: Jupyter designers, feel bad about yourselves.
Andrey: No, no. Well, there are alternatives now that are, better at this.
Seth: What should our audience look, go to, Andrey?
Andrey: Well, I think at this point, the AI should be creating a UX for you with the results.
Seth: Like an app.
Andrey: And then you go back and forth with AI. Like writing literal code at this point, I mean.
Sarah: Yeah, I think that an interactive paper is a much better idea. I think that we are stuck thinking about journals and their paper constraints because they get physically published still, and that’s not necessarily… Yeah, Seth has a lot of them.
Seth: I’ve got my Journal of Economic Perspectives over two decades down here.
Sarah: Yeah. I think that there is a lot of room for innovation here. At the same time, I think in computer science, they’re just putting stuff on arXiv. And their problem is a little bit different from ours because in computer science, if you build a package or a tool that’s valuable, then it’s gonna rise to the top on GitHub. You’re gonna understand the tool. You’re gonna access a tool. You’ll use it. I don’t know if it necessarily works the same way in economics or in information systems. And so I think that part of it is, yeah, I would love to think through what the future journal looks like in our disciplines.
Seth: I’m gonna tell you the future. Everyone just posts everything as working papers that are continuously updated. Everyone just posts public comments on different working papers. This is Seth’s, review of Acemoglu’s latest paper.
Andrey: This is Seth’s AI’s review.
Seth: [laughs] This is Seth’s AI’s review. Seth, the human centaur interaction with his AI review of Acemoglu’s latest paper. And you can you will have your favorite reviewers, and you will tell your AI, “Tell me what’s the best paper on optimal tax policy according to my favorite public economist,” right? And that’ll be how we engage with it.
Sarah: Like Goodreads.
Seth: Like Goodreads, exactly. My favorite social media.
Andrey: The journal system kind of has so many different roles, and one of them is just, a bean-counting thing for promotion committees at universities. And one of it is some sort of level of certification, which we know is highly flawed. And I think another one is just, an attention mechanism, right? If something is published in the QJE on a particular topic, that’s like the field has deemed this to be a very important thing that everyone should pay attention to, like that R Epstein can co-author paper that you said in yours. I think that was a QJE, but maybe that was AER. I don’t remember. But, one where, it coordinates the attention of the field, and that’s really important because, common knowledge helps us coordinate and make sense of the literature. And so those are bundled for pretty historical reasons, right?
Seth: Prestige is often works that way, right? That you have path dependence in it.
Sarah: Yeah. I think the area that I’m most interested in is this veracity or certification component, because I think that it’s harder and harder to figure out what is good science not for us, but for the public. Yeah, I think that there’s so much on social media where it’s like, “Is this actually true?” and so if that’s the case, then I think that peer-reviewed research actually ends up making a bigger difference in that context. And so I don’t know what I see as a substitute for that for the public.
Andrey: I so disagree. Peer review has very little ability to certify for the public. The public has— can’t tell the difference between two journals. The New York Times and other media outlets breathlessly cite garbage all the time, oftentimes in very good journals like, New England Journal of Medicine or whatever. Just BS correlational studies, and the public doesn’t trust it already.
Seth: Where something that’s more like a Goodreads rating, something that’s like, “This paper was rated 4.5 by, the 100 voters of the AEA,” that’s a lot more transparent and readable to the public, I think.
Andrey: Well, it’s interesting. In general, the public interaction with science is very superficial, so it’s kind of a lot to ask. I’d say it’s like the, a bigger issue might be people are making decisions, grant-making bodies, investors, even, potential employees joining a company. Is the science based on this solid or is it like Theranos or whatever? You know, this sort of stuff, I think there’s some value, but I don’t think it— [laughs] it’s— we’re so far removed. [laughs]
Sarah: At the same time, I do think that the public is very important in this question because the public is what is primarily funding science.
Seth: I don’t see why we can’t move to a prestige validation system that is more open and democratic and preserves the information that’s thrown out in these reviews, and we just, yeah, we move to it’s the big pile of science, and you will still get coordination because it’ll still be the highest ranked thing. They’ll still be the thing that’s endorsed by the Nobel Prize winner.
Sarah: Yeah. I do think that understanding comments of reviewers more broadly and be— having that as visible to others would be pretty valuable. I think that, we don’t have this in our workflow, but I think it would be cool if, the AI review was posted, when the, when the paper was published, and these were the concerns that were described, and these— this is how they were addressed. Somebody could also run Refine themselves on a paper and look at it, and I think that’s increasingly going to happen, that, people are going to evaluate other people’s research using some of these tools. We’re seeing it online already.
Andrey: One thing that we have to remember in the background is the price of intelligence is falling precipitously, and I think the bottleneck is more like most people just don’t care about our research. They just don’t care. If you find, the 100 people or the 1,000 people that care about the research, it’ll be very cheap for them to at least get a quick pass AI to, read the paper and, what are the main concerns? Is this argument convincing? And so yeah, it could be useful to have an AI artifact attached to the paper that kind of is a critical reading of it, but it’s also gonna be trivial for someone to just generate that themselves. It’s not gonna be expensive.
Seth: And especially ‘cause the reviews are gonna get better every year, right? So who cares about the AI review from two years ago?
Andrey: Exactly. That’s right.
Faith and AI: the Pope, the Imam, and steroids, sneakers, or a coach 54:36
Seth: The last question we wanted to ask you about, Sarah, is, it’s been really exciting getting to know you as a friend and kind of sharing with each other about our faith traditions. So we recently read the Pope’s AI encyclical. … Pope’s got some opinions about how AI should integrate into society. Has the Ismaili Imam made any similar AI statements? And, if not, what would you recommend him to write about AI?
Andrey: Who is the Ismaili Imam?
Sarah: Yeah. So, let me, start with a quick introduction.
Seth: Oh, give us some background context. Yeah. Okay.
Sarah: I’m Sarah. I’m a-
Seth: Hi, Sarah. [laughs]
Sarah: …A Shia Imami Ismaili Muslim. So, there are two interpretations of Islam, Shia and Sunni. Within the Shia branch, there are different groups. One of the groups is the Ismailis. The Ismailis have a living hereditary Imam that is a direct descendant of the prophet. And so, that is Ismaili Imam. He has been, speaking to his congregations, in a variety of different contexts, and AI has absolutely come up there. I have not found a publicly available speech, but I did find a publicly available graduation speech. It’s a bit of a traditional graduation speech, in the sense that it’s inspirational, it encourages people to use new tools wisely, and encourages people to care about the wellbeing of their communities. And I think that those are all things that we should be thinking about. The Pope talks about very similar things. If I pull up my notes, “We believers must and can choose which projects to work on and in what manner so as to safeguard and value the grandeur of humanity that has been given to us as a gift.”
Andrey: So, why aren’t you working on AI safety, Sarah?
Sarah: We are exemplars for how to use AI productively in the world, and I’m certainly taking that responsibility seriously. I talk a lot about this analogy, the steroid sneaker coach analogy. Have you guys heard this one?
Seth: Give it to us.
Sarah: So, you can use AI in three ways. First, as a steroid, which means improving your performance in the short term for long-term degradation. You can use AI as sneakers, which makes you faster but doesn’t fundamentally alter your performance. And you can use AI as a coach, which involves improving yourself. And so I think about this a lot because I think we all have to ask ourselves how we’re using AI, and we want to be using AI in the coach and sneakers manner as opposed to in the steroids manner.
Seth: Now bring us back to religion, and the Pope wants steroids and the Imam wants sneakers. Now-
Andrey: I don’t think that’s true. [laughs]
Sarah: I think both the Pope and the Imam care about marginalized communities and encouraging marginalized communities to be supported in all sorts of ways. I think both the Pope and the Imam care about connection and connection being fundamental to, our identity as human beings and using these tools to, for example, speed up processes such that you can spend more time on connection.
Seth: Speaking of using tools to speed up processes to spend more time on connection, I have noticed rabbis, in around here starting to use AI both for, efficiency tasks and for writing sermons. Have you, in your, community, have you seen AI sermons yet?
Sarah: So, we don’t have sermons in our community. It’s not surprising that people are using AI to brainstorm. It, the question is oftentimes about the delivery matters and the truthfulness of the experience matters too. And so, if you can use AI to tell a story better, then that’s great, and for it to connect more with your congregation. I do think that the encyclical, I really enjoyed it. I only read chapter three. I hope people read these things and realize that other people want them to focus on humanity and think about what that means on a day-to-day basis, right? We’re really lucky we get to spend time as part of our work or not with one another talking about these things. I think that it puts a smile on my face because we’re, I mean, we’re friends, right? And, this is our job, and we get to spend time with one another and think about these things. And, I don’t know if a lot of people necessarily get to do that. They don’t get to have deep connection. They don’t get to, get good and bad feedback on their takes, because they’re not connected to people. And so, I think that, yeah, I don’t have any hot takes, but go find a friend, share your hot take with them if you have it.
Seth: There you go. That’s the hot take-
Andrey: Oh, yes. There it is
Seth: …Is have your own hot takes with your friends. Oh, I love that.
Andrey: All right. Well, this is a great time, I think, to wrap up. Sarah, it’s been a pleasure. Thank you for coming on the show. Good luck with the new job.
Sarah: Thanks so much. Really appreciate it, guys. Good seeing you.
Andrey: Well, thanks for joining us for another episode of “Justified Posteriors.” As a reminder, we have written down all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out. And if you liked the episode, please share it with your friends.