Can political science contribute to the AI discourse?
Reading "AI as Governance" by Henry Farrell in the Annual Review of Political Science
0:00Seth: Welcome to the Justified Posteriors podcast, the podcast that updates its beliefs about the economics of AI and technology. I'm Seth Benzell, finding myself the complete opposite of a political scientist, as I have a remarkable amount to say about AI, coming to you from Chapman University in sunny Southern California.
Andrey: And I am Andrey Fradkin, trying to eke the last bits of cultural production out now before the AIs take over this crucial role, coming to you from Cambridge, Massachusetts. So yeah.
Seth: Squeeze that cultural fruit for all of its juice, Andrey.
Andrey: Yes. Yes. So what are we talking about today, Seth?
Seth: We are venturing outside of our core domains of economics and computer science into the wild, dangerous world of political science. It's, kind of exciting for me, going back to my roots. Did you know I was a political science minor, Andrey?
Andrey: I may have known that, but did you know I was also a political science minor? [laughs]
Seth: What? [laughs] So we're both kind of coming into this with a little bit of interest. So did you-- were you mostly interested in kind of international stuff or, domestic issues when you were, taking your pol sci classes?
Andrey: I was, I was interested in political economy related issues, not necessarily international ones. What about yourself?
Seth: Oh, I was super into these international, global issues. I was president of Tulane's Model United Nations, Tulane Organization for Global Affairs, TOGA. If you're out there, giving you a shout-out. Everyone sign up for Tulane Model UN. Damn, that's throwing me back. But yeah, I feel in some ways like the legacy of an older, more optimistic age, right? I feel like when I was taking political science classes, we were coming off of the US' golden unipolar moment, and there was a sense in which universal peace and pos-prosperity was possible. We were gonna bring China and India into this beautiful, prosperous international system. Russia was, tired of invading Eastern Europe and was gonna just become you know, France. And unfortunately, that golden world that we thought we were building towards, I don't know, were we too idealistic? Were we naive? It seems like the promises of political science and international relations that I once was made-- once were made to me, the cash has, failed to clear.
Andrey: Well, wow. That was, that was very poetic, Seth. I do think that, political scientists are not known to agree over, many things, such as the ones, that you were saying. So I think, I think maybe, that optimistic view of the world was, a cultural product, if you will- ... Of, various media environments. But I actually don't think it's, it was the predominant view of all political scientists. I think, there were still many international issues that, at play e-even, when we were, still taking our undergraduate classes.
3:36Seth: Right. Right. Well, I remember very, very vividly the live issue at the time was, can you convince China to join the international system? I think that was the big thing looming over the world. That and, international Islamic terrorism, right? Which is-
Andrey: Yes. Yeah. The terrorism issue was a really big, yes
Seth: ... Like, obviously continues to be a thing, but- ... It never really felt like a fundamental threat to the international project. People talked at different points. I know. It felt different in 2001. What can I say? You can't-- these days, you're listening to it, you don't understand what it was like. Um-
Andrey: Well, I mean, I mean, I remember a lot of debates about what system was best for passing good legislation, or if par-parliamentary system was better or not, for example. I remember, also reading some more critical things, like some Foucault. And also, that was a-around the time where you really started seeing the positivist turn in political science, where political scientists became a whole lot like economists in many ways.
Seth: Right. All-- and in fact, this essay that we're going to read, maybe this is a good transition point, argues that political science has not become economics-y enough. This is... People say that e-economists have physics envy. This is a political scientist, that seems to have some econ envy.
Andrey: Yeah. Well, yeah, let's kind of introduce the essay at hand. So the title is AI as Governance, and it's by Henry Farrell. And it's published in the Annual Review of Political Science, which, we have a similar thing in economics, which is usually a place where one puts kind of review articles that are kind of summarizing a literature. And it's very interesting for this type of article in that, y-you know, it kind of starts out with saying, "Oh, there's not that much literature." Like, political scientists have kind of been- ... Ignoring, AI, in a surprising way. So, yeah.
Seth: Yeah. Let's just throw out-- Can I, can I throw out like prior zero before we get into our serious priors? The prior that there is, a, not a lot of political science literature on AI is roundly defeated by this own [laughs] by the own, lit review on political science in AI, which is manages to, get up to four pages of citations, admittedly many of them to econ or sociology. But, you know, talk about this T-shirt seems to, lead to questions that have been answered by the T-shirt.
Andrey: No, no, I-- So yeah, I kind of disagree with you here, Seth. Maybe we should have made this a prior. Yeah, most of those articles are not political science articles. Um- ... And I don't see political scientists being very prominent in any of the AI conversations, to be frank. I don't know, can you really think of a political scientist who, we are paying attention to very closely?
6:36Seth: The people I think of who have the best political AI takes would be kind of the general wonky guys that we would put in kind of our econ popularizer camp, right? We think about Abundance just came out, Ezra Klein, Neil Thompson, kind of, the center lefty technocrats seem to be writing a lot of good stuff about this. One name that comes up in this essay that I wish was more highlighted and engaged with is a friend of the show, Glen Weyl- ... Is one of the cited people. [laughs]
Andrey: So you're gonna cite an economist as an example of a political scientist we should pay attention to. [laughs]
Seth: Well, and he doesn't think of Does he call himself an economist these days? He wants to be-
Andrey: Oh, no
Seth: ... A political scientist.
Andrey: They-
Seth: This is inside baseball. Sorry. Go ahead. Sorry.
Andrey: They try to, they try, they try... [laughs] You try to leave, but then they pull you back in, as they say. But anyway.
Seth: So maybe, but yeah.
Andrey: Yeah, we love you, Glen. No, no offense on your preferred a-affiliations these days. [laughs]
Seth: Right. Okay. But, so Glen would be an example of someone who's maybe coming from the econ camp who's saying, "Hey, we can use these tools, these market design ideas, instead of to do incremental improvements and, sell our bandwidth for 10% higher at auction. Rather, we can use mechanism design to imagine radically different communities." I encourage listeners to listen to his book, Radical Markets, which is hardly definitive, but is a beginning of what are some things that we can do now with technology that maybe weren't even possibilities in the past.
Andrey: Well, that, that was alongside, which I hope listeners don't mind. But let's get to our priors, actually.
Seth: Okay, so this is an essay about... What's the essay about? It's an essay about, AI as governance, and how do we operationalize that as priors, Andrey? Bring me there.
Andrey: Yeah. So the first prior that I had is kind of the sweeping prior, which is, AI will lead to new forms of governance that are so different from the current ones that we'll kind of need to reconceptualize, how we think about governance altogether. So that's kind of, the first one. And then, the second one is a useful way to think about the effects of AI and governance is as a new cultural technology, used in battles over political influence. So imagine, back in the day, you might have competing radio programs or TV shows, but now you have AI-generated artifacts. So yeah.
Seth: So the theory would be something like, books give you enlightenment, democracy, the radio gives you fascism, AI gives you question mark. Is that fair?
Andrey: Yeah. Yeah. That's one way to think about it. But I don't, I don't, I don't see a deterministic claim here. We don't know. I don't, I don't even think even in the past, there is deterministic. But yes, this new technology will create a new cultural battle, essentially.
9:42Seth: Right. Okay. So let me start by telling you how I feel about the first claim- ... Right, which is that we're going to need to radically reconceptualize how we do political, science and political economy. I guess the first thing I would say is kind of we already have a word for the study of sociotechnological systems for governance, right? And we call that cybernetics, which is a really kind of cool 1970s word from early AI days, drawing on ancient Greek, the term for steersman- ... 'cause we're gonna kind of steer society with these sociotechnological insights. So I guess, what I would say is I thought that political thinkers have been thinking about technology driving society for a long time. I came in sort of surprised that we would need to, start again from scratch on this. But having... Yeah, so I came in maybe not convinced of this prior. I would say I came in, 60, 70%. Don't we already have cybernetics, Andrey? What is that missing?
Andrey: Yeah, I mean... [laughs] Yeah, I'd say once again, it's a, it's a prior, we, the conceit of the prior posterior, right? Yeah, I would say when I, when I-- Before I read this essay and I thought about it a lot, my hunch would be that, yeah, once we have artificial super intelligence and it just tells us what to do, then yeah, we should probably rethink how we think about governance. But, short of that, it did it did seem kind of more like continuous, refinement of government. With the caveat that, if we were to start from scratch, which, we rarely ever do, the various revolutions accepted, we would not design the government the way, the way we have now, and we would probably use a lot of, automation technologies in various parts of the bureaucratic process.
Seth: Percent.
Andrey: Yeah, I'm having a hard time with percent here, Seth. I don't, I don't even... I can't do it.
Seth: Disqualified. All right, cool. Well, when we do our annual roundup of whose posteriors were most justified- ... You'll just have to default on today's episode. All right. So, the second question you asked, was, is the right way to think about AI shaping governance as through changing the way we have our cultural debates? That, I think, is actually a pretty cool framing that I hadn't really considered before. I'll tell you kind of where I end up at the end of this essay, but going in, I didn't really think of it that way. I would say maybe, 40%, right, was kind of my attitude coming in. Well, but loosely held, as you can see, and you're 50%.
13:00Andrey: Yeah. I guess I would put mine lower here even, 'cause just 'cause I wasn't thinking about it this way, right? I was primarily thinking of AI as a tool, for existing processes rather than a cultural technology. Clearly we've discussed, persuasion in the past on the show, and so it's certainly not, something we hadn't considered. But when I think about all the things that are important about AI, I've never been one to believe that persuasion is that important. [laughs]
Seth: Okay. So, do you wanna put a percent on that or are you gonna demure again?
Andrey: I'll do, 10%.
Seth: 10%. 10%. This man's ready to be persuaded.
Andrey: It's a pretty... I'll admit, it's a pretty... It's a vague claim, but when you think about it, the main thing about AI is that it's gonna create different types of media to... That, that to me, seems unlikely.
Seth: You could be persuaded of it, but you're not starting there. Okay, fair enough.
Andrey: Yeah, I'd certainly do not start there, yeah.
Seth: All right. Are we ready to move to our evidence? Do, do. All right. So, what we're faced with here is, again, a little bit different than our normal, positive turn at the papers that are going to, show you some evidence and make, hypothesis to test it. Rather, we get sort of a three-part essay. In the first part, we get a definition of what AI is and what governance is. Unsurprisingly, I'm gonna like their definition of governance, Farrell's definition of governance a lot better than I like his definition of AI. [laughs] Then we're going to proceed to talk about AI as a tool for governance, and then finally we'll talk about AI as governance, this pretty, abstract idea. But before we get into that, we already talked about kind of the initial claim that was made, which is that political scientists have said remarkably little about AI. Yet here we are remarking, Andrey. [laughs] Do you have any comments on this, intro material where, Farrell... Is it, sorry, is it Farrell or Farrell? Where Farrell, bemoans the state of contemporary political science? Or are you ready to jump into the arguments?
Andrey: Yeah, I mean, the, I think the only thing I would say here is that I think I agree with him a lot more than you, but I think it's just a function of, the fact that a lot of modern political science is very empirical. And we've already talked about the challenges of, studying things that haven't happened yet as an empirical researcher, right? And especially without, a bulwark of formal theory. And so my sense is that he's actually fighting a different, slightly different battle here altogether, right? Which is this battle that maybe political scientists shouldn't just narrowly write papers asking surveys, survey questions of people, and maybe they should, do more of that big picture thinking that they had done in the past.
16:19Seth: This is so interesting because it feels like we had very different political science experiences, right? [laughs] It sounds like you were taking these, basically poly econ classes about political equilibriums and, testing, positive hypotheses, and I was taking these, IR classes that were about why do countries fight? Is it because of the incentives of leaders? Is it because of broad social forces? It seems like the political science that I know about is all of that Fukuyama, you know-
Andrey: But Seth, I guess what I'd say, I actually don't think that's true. I just think that- ... That was the old guard that no longer exists, or they're very old now.
Seth: Yes. Yeah, Nye.
Andrey: So it's not our class experiences.
Seth: They're dying out. Dude, Joseph Nye, Joseph Nye, history of, US soft power just passed away, RIP in peace.
Andrey: Yeah. If you look at people our age who are political scientists, they all do this sort of positive, not all obviously, but the predominant do this sort of work. And so, oftentimes I encounter political science, as a field that uses and invents, new methods in, causal inference, for example.
Seth: Right. So this is, this is political science in a, in a... You're, we're encouraging them to maybe go back to an older mode. Maybe this is a, isn't a reinvention, maybe it's a retreat. Retreat's the wrong word. Okay. So all right. So that's our where we're coming at this. I may be more familiar with a different slice of political science than you are. Now we enter this question of what is AI? What is governance? I'm tempted.. Why don't I take on governance? That's okay. I thought the definition was pretty interesting. They define governance, Farrell defines governance as a system for bringing in complex information, processing and representing it, and then helping people coordinate based on that representation, right? So inside of governance is gonna be everything from the king tells you what to do, to the market system aggregates demand information and comes up with prices, and then we make decisions individually based on those prices. In this system, those are both governance. So governance is quite broad here. It's any- ... Sort of complex social coordination mechanism.
Andrey: I guess he would say very clearly in this essay, he says he doesn't have a definition of governance. It's a loose metaphor, right? It is a, the value of this approach to governance is not at a, that it provides precise definitions, let alone testable hypotheses, but rather broad heuristics.
Seth: I treat governance as an umbrella term for the large scale systems- ... For processing information and social coordination, right? It's a, maybe it's not a, it's not a definition, it's an umbrella term.
Andrey: It's an umbrella.
Seth: All right. Fair enough. [laughs] one-
19:23Andrey: Kind of like institutions, if we're- ... Something that we're quite familiar with.
Seth: Oh, God. God forbid on an exam somebody asks you what the difference between an institution and a governance mechanism is. [laughs] my, okay, it's for a really hard aside. My wife really loves these sort of, psychological murder detective TV shows, right? And so one term, two terms of art that come up a lot are, triggers and stressors, right? And so, a trigger is something that might cause a potential murderer to murder. Whereas a stressor is something that might cause a potential murderer to murder. [laughs] So anyway, triggers and stressors, institutions and governance. All right. That may have been too much of an aside. Returning to the paper, one quote I found, very fun here was explaining how everything is kind of governance. The price mechanism, bureaucratic categories, and representations of the democratic public are simulations or very lossy core screenings of irreducibly complex systems.
Andrey: So let's dig into that a little bit. Let's just think about what that means, let's say, in terms of democratic governance, 'cause I think that might be a useful analogy. There, if we, if we read the founders and various political philosophers, we're kind of thinking about this ideal where we kind of know what every single person in a populace wants, and we kind of aggregate that, in a useful manner, and then we get some good outcomes. And maybe we want some minority protections, and maybe we want some expertise in there. But generally, there's a conceit that we're kind of representing kind of what people would want if they thought really hard about it and were fair.
Seth: Right. That's the populist conceit for democracy.
Andrey: Yes. And, you can see how that's a lossy one because, clearly, who wins in, let's say, the House of Representatives or, who is put on the Supreme Court, you might not think that that's a pretty great representation of the people. But up to debate, obviously.
Seth: If you, I mean, I, Andrey, I'm surprised you didn't immediately go to Arrow's impossibility theorem and the old results in economics that there is no such thing, or there is not necessarily such a thing as the will of the people, that the people may be irreducibly divided and hopelessly irrational in their aggregate preferences.
Andrey: Yeah. I guess there are formalizations of this, but I was even thinking about, on a very kind of observational level of just, our political institutions have all sorts of lossiness, if you will, to them, stemming from, imbalances and, resources and attention and, various-
22:33Seth: Right. I have to combine all of my-
Andrey: ... Moral hazard issues. You know? Yeah.
Seth: I have to combine somehow all of my preferences into this ballot that- ... Gives me three options, right? Of course that's lossy, right? And so are, maybe what you're building towards here is this idea that an AI agent-driven democracy would a, be able to much more detailedly, take in our preferences and aggregate them. Is that what you're building towards?
Andrey: Yes, exactly. But I guess I wanted to lay out the, where the lossiness comes from in this sort of system. And, of course, markets are a different way of, organizing society, focusing on economic, factors, perceived through the price mechanism. And once again, we can think about various ways where sometimes we might think that competition isn't working as well as we'd like at, efficiently allocating resources.
Seth: Right. And so now, I mean, if I can be even more postmodern for a second, let's make the postmodern move, right? So we're gonna describe these governance mechanisms as producing simulations, right? What's a simulation? A simulation is, there is some underlying reality that I'm trying to represent in a artificial way, right? As we all know, father of, or father of the Matrix, Baudrillard, has told us that we are moving beyond simulations into simulacra, so simulations of things that never existed in the first place. You might think of the bureaucracy itself as a kind of simulacra if you think of... Now I'm gonna be, completely out of my depth here, but, in China, there's sometimes this idea of The heavenly bureaucracy that our earthly bureaucracy imitates, right? So if you're imitating a thing that may or may not be real, you'd have a you'd have a simulacra, not a simulation. And in the same way, you might worry that as we move to this more kind of AI agent-driven, detail-preferencing, auto-generation of content, we're-- the simulation that is the governance mode has less and less to do with what real individual people maybe hypothetically in aggregate would've wanted, and becomes more and more a simulacra.
It's a simulation of a hypothetical person that never existed, that exists in order to achieve, I don't know, question mark. In Baudrillard, [laughs] you know, it's like you can't achieve anything 'cause nothing's real, right?
Andrey: I am impressed by your ability to bring up Baudrillard here, Seth, after we read it and agreed that it was 99% nonsense.
Seth: Oh, it's 100% nonsense, but it's beautiful, poetic nonsense.
Andrey: [laughs] Which, is a cultural, technology, if you will. Um- Okay. But before we-
Seth: Don't read Baudrillard, just watch The Matrix. That's the answer.
Andrey: Yeah, The Matrix is great, yeah. I guess what I'd say is, before we move on to this cultural postmodern take on all this, let's think a little bit about what AI as a governance technology would mean here. And here I was, looking at some of the recent literature. There's some folks at DeepMind, that have been interested in kind of AI as a tool for democratic consensus building. So maybe you have an AI agent that's kind of a mediator between people with diverse beliefs, that kind of helps them to agree on a statement or a particular viewpoint or a particular- ... Policy. Um-
26:14Seth: Arbitration bot.
Andrey: Yeah, exactly. Or you could think about, this is, in some work of a friend of the show, John Horton, where, people might have very complicated preferences, but it's hard to state them. But at the same time, if you state some of your preferences and tell an LM that, the LM can reconstruct more of your preferences. And so then we can have, maybe we can relax the constraint that people are gonna vote over these, binary things like candidates. And maybe there's kind of a different sort of expression of democratic, opinion, that is AI-enabled. Maybe you designate an AI on your behalf to vote for you, for example. So there's all sorts of kind of interesting technologies there. And then there's kind of more mundane things, which are still, tremendously important. But like, instead of having a bureaucrat that decides, whether you're eligible for Medicaid or not, you have, an AI and, that, that AI may be, this LM-driven thing that's kind of fuzzy. It's not like a set of rules. It's actually making judgment calls depending on what its conception of fairness is. But maybe both, maybe we as a society agree to hand that over to the AI because it's, more objective, in some way than a human bureaucrat, or it has more attention to detail or, so on and so forth.
Seth: Right. And so now, so the distinction that, Farrell wants to make is between AI as tool of governance and AI as form of governance. So just to roll through some of the examples you had, Andrey. The first, arbitration bot. So we have a dispute, and then we're gonna have an AI try to help find us a compromise. I would call that form of governance rather than tool of governance. I guess if you were ordered by the court to do it would be a tool of governance, right?
Andrey: Yeah, I think it's closer to form. I don't think it's, obviously a clear distinction here. Yeah.
Seth: Okay. Simulated polling, the John Horton simulated preference, extraction. Is that a tool of governance or a form of... That's a tool, right?
Andrey: Well, I mean, it could become a form of governance if, it's kind of hyper-democracy, if you will, right? It's kind of like direct democracy, but maybe, has some advantages.
Seth: A simulacra democracy.
Andrey: Yeah. Yes, exactly. Something like that.
Seth: Kind of delegated polling. So this idea that I would have my own personal AI agent that can vote in a kind of complex parliament on my behalf. That feels like governance, not a tool of governance, right? That sounds like we have a new system. Okay, cool. I will now say that each of those three very fascinating ideas do not appear in this essay. [laughs]
29:16Andrey: Too specific for this essay, Seth.
Seth: Too specific. Too specific. Let's spend 300 words about whether, LLMs are racist instead. Okay.
Andrey: Yeah. So I mean, let's just flag it, right? So there's a pretty big discussion of bias, both kind of standard bias that we might be thinking about, from everything, all the discussions of bias that have existed in society based on, socioeconomic status or gender or, race or whatever. But he also talks about cultural bias, which might be something different. Maybe, biased against people who, I don't know, like anime or something like that. Uh-
Seth: That's fair. Yeah.
Andrey: But I do, I do think that, to me it's not that bias is not an important topic, but there is a tendency to frame many, very many interesting issues as issues of bias rather than maybe more helpful, framings of various topics to study
Seth: Right. So let me give you the quote on that, that I found the most interesting. So he, Farrell asks the question, "Should LLMs correct for a dominant/hegemonic view that reinforces existing forms of inequality and oppression? Should they instead reflect some unfiltered version of the cultural content that has been scraped and fed to them on the justification that this reflects some raw version of the marketplace of ideas? These are profoundly political questions. Even a few political scientists or theorists have taken them up so far." and the essay goes on to talk about all of these things we've already mentioned in our algorithmic bias episodes, right? That you might... There might be irre-irreconcilable trade-offs between different methods of bias, types of bias that you might have to take on, right? And so I think a more interesting essay would have... Farrell, we appreciate what you got. This is a good essay. We definitely were inspired by it. I wanna read the essay that's, taking on this challenge of, what, here's a bunch of concerns we have about LLMs. How, what's the, how does politics help us get a better, to a better place there? Maybe that's a political econ essay.
Andrey: Yeah. Yeah. Or, alternatively, how do we conceptualize this? We have these different model providers that have pretty different philosophies on, what the system prompt should be, right? You know, we have your Groks that are kind of anything goes, but maybe a little, a little special Elon term. And, we have, we have your Anthropics. They're forming a new constitution over there. We have OpenAI. Let's... Who knows what OpenAI is doing. I have no-
Seth: Yeah. We've got Google, which, uh- ... God forbid we had German Nazis in our pictures.
Andrey: Yeah. And, but what's also interesting is that some of that seem, is pretty clearly also unintentional, which is kind of a key issue here, right? It's not... Clearly, they figured out that that was not what they want. So with their, with their image generation.
32:25Seth: Right. So yeah, so to, for, to bring people on board who don't know what we're talking about, there's kind of a famous example of, Google image gener-de-generations wanted to make sure that it had, good diversity of people whenever it generated a group of people, rather than, reinforcing implicit biases that, doctors or lawyers might be of a certain race. And what they ended up with is, no matter what you generated, you would get an ethnically averse, diverse array of, whatever, Nazis, communists at the turn of the century, whatever. Whatever group that you had in mind.
Andrey: Mongols.
Seth: Mongol invaders, right? Ethnically Not that the Mongol hordes didn't take on a lot of ethnic diversity over the course of their conquests, but, you can see how this ended up in a silly place. Why am I bringing this up? That was a spectacular example of de-biasing gone. Nobody's happy with this, right? That was... [laughs] But that's the kind of politico socio question that theoretically political scientists should be able to help us with, right? That's kind of what Farrell's arguing.
Andrey: Yes. Yes. And I think here's kind of where maybe we can make the move to the cultural technology. So, yeah, so once again, to reiterate, when we have new technologies such as TV or even cable TV, that creates new media. That media is used to, support, either explicitly or implicitly certain political forces, right? And the question is AI that? And what does it mean for AI to be that? And I think, just spitballing, I think a very, obvious way in which that's true, but it has nothing to do with LLMs actually is just algorithmic news feeds, right? Which are, we've already discussed on this show. They-
Seth: Yeah. So there's a good amount... And I will say, to give Farrell credit, he does spend a good amount of this thinking about AI governance on digital platforms. He doesn't have that much to say. Community Notes never comes up, which is something I would love, to see discussed in this essay. But, the author is aware of AI governance happening in platforms right now.
Andrey: Yeah. And to be, even without Community Notes, right? We have, even just the decision of which post to flag and send to human reviewers. All this, all this stuff is governance, and books, many many books have been written about it. Many papers have been written about it. I think the question is, the LLM version or the agentic version of AI, is that gonna produce new cultural artifacts? I think one way to think about that is, we can generate texts from scratch. That could be persuasive essays. It could be movies. It could be podcasts, right? And maybe Seth and I, we want to propagandize the value of, I don't know-
35:26Seth: Our podcast
Andrey: ... Economics education and-
Seth: We're propaganding the podcast.
Andrey: Our own podcast, for sure. Uh-
Seth: Thank you.
Andrey: And so we create a bunch of, AI-generated artifacts that are, we're gonna unleash into the memosphere, and they're gonna capture some amount of attention. And, as a result, if we win out in this cultural battle, we're gonna see more, prominence, and economics podcasts are gonna have, more of an influence in society.
Seth: Yeah. And society will be healed.
Andrey: Which-
Seth: Presumably. [laughs] Right. And, that's, I mean, that argument sounds pretty right to me, right? It seems like you only get personal personalized autocracy with mass communication. It seems like, modern dictatorship is only possible with radio and television, and then, and then what? Right? So what is the form of governance which is unlocked by mass personalized communication? The positive vision is something like we are able to use our AI systems to explain, voice our opinions in much more subtle ways. That's the positive spin. The negative vision is that, we get overwhelmed with slop, is that the public discourse becomes super low-cost invented material, invented people, astroturfed political campaigns of imaginary people, who are out there marching in the streets, but there's nobody's marching in the streets. And that whoever is able to control all these simulated agents can then control the perception of what is popular, and then control the reality of what happens.
Andrey: Yeah. So, aren't we there already, Seth?
Seth: [laughs] Well, that's the thing, right? As you said, this doesn't really engage with the actual positive literature that, that much. Let me talk about two ways, pieces of evidence we have on whether that dystopia is live. The first is, a paper that we reviewed on this podcast. Look it up, the episode, "Did Meta's Algorithms Swing the 2020 Elections?" One of our best episodes, I might say. And there it, in contrast to this essay that wants to distinguish between, purely technocratic, decisions of informed bureaucrats with political decisions, and argues that these political decisions... Sorry, argues that when tech platforms make any sort of decision, it's inherently political. Well, we just saw [laughs] an example of the algorithmic newsfeed in this super-duper precise study, at least once, at least in 2020, seemed to be apolitical. It seemed to mostly just be about getting people to be on Facebook longer.
38:30Andrey: Well-
Seth: So that's the first piece of Yeah, go ahead. [laughs]
Andrey: Well, I mean, what... I would not read that paper that, that it was apolitical. It changed the distribution of really bad slop content actually, if you recall. The algorithm got rid of a lot of low-quality news content, which I would suggest is the sort of content that we might that people expect AI to produce, right? Just a lot of, lot of fake news. Sensationalist, not so true stuff. Um-
Seth: But it ended up not making a difference. Even if it did that, if it did the good political thing, it seemed to have not moved people's politics.
Andrey: Well, I think, I think we discussed why that's not actually something we should learn from that study, right? And I do think that this is where a macro perspective is really important. Opinions are... I'm gonna go back to this every time we talk about this. Opinions are formed in a social community, and so if you yourself get treated, and you're the only one who gets treated out of all your friends, the effects are gonna be very tiny. And, 'cause just we don't change our beliefs outside of a social context unless we're like philosophers or something.
Seth: Right. Most of us just go with the crowd. The second piece that I was thinking about here was the research that we've already gotten from friend of the show, Bo, who has been working on this question of the kind of the more narrow question of how do AI-enhanced resumes and communications kind of gum up the information revolution process that happens in job applications. And I know, Sarah Bana at the Digital Economy Lab, lots of people are kind of at the econ side working on this question. Farrell, cites himself as making an argument that LLMs are extremely well-suited to the automated performance of organizational ritual, such as the production of personal statements and performance reviews, blah, made to generate, leading to inferior knowledge and lower trust. So, the political scientist here is starting to catch up to this econ concern that communication is gonna get degraded because we've made explaining yourself, less costly signal perhaps.
Andrey: Yeah. And I guess following on to that, I recently read a piece from the Center for the Governance of AI, which is an organization that seemingly [laughs] should be relevant to this essay, called, "What Role Should Governments Play in Providing AI Agent Infrastructure," right? So if we think that-
Seth: Is this Gov AI? Is this the Oxford group?
Andrey: Uh-
Seth: Or is this something else?
Andrey: I think it's Gov... I think it's the Oxford group. Yeah. Uh-
Seth: Love those guys. Hive Sam Manning into the team, Phil.
Andrey: Yeah. I'm not 100% sure that they're who you're talking about, but, um-
41:32Seth: [laughs] Okay. If not, knock 'em low.
Andrey: But nonetheless, they seem legit. Anyway, so they... So kind of this question is like, let's say, the government is being forward-looking about what to do about AI systems. Well, one thing to do is to just let the market solve AI systems, right? But you might think that if there are these unpriced externalities, then maybe, the government needs to get involved, and the question is kind of what are those unpriced externalities? And kind of already here suggesting that maybe something to do with the low costs of cultural product production and verification of who produced what content and which content is trustworthy and which one is not. Maybe there's a sense where, the market's gonna try to provide for this, but maybe it's gonna have a hard time coordinating, and therefore the government might help to coordinate that, somehow. Or maybe, or maybe it's gonna-- might mandate some transparency and disclosures that might be helpful in this situation. Or it might issue credentials that could be used, by individuals and organizations to verify certain things.
Seth: Andrey, thinking about this like an economist, like a political economist, right? Here's a tool, it's lowering costs. Let's think about mechanism design in order to overcome that cost. I think what Farrell wants is not that. I think Farrell's happy to delegate those questions to computer scientists and economists and et cetera. The question he wants to ask is, one level deeper, right? How do we come about having a government that wants to solve that problem or not solve that problem? How do we go about, establishing the desired data for the solution of that system, right? And I'm not sure that those are AI-- I mean, those are AI questions insofar as the social decision could be shaped by AI, right? But in some ways, they seem kind of pre-AI, right? In some in some ways, it just sounds like, okay, what do political, politically powerful people want? Okay, they'll do that with AI now, right?..
Andrey: Yeah. Yeah. I agree it's kind of a mechanism design approach, but I guess maybe I'm reading the essay in a different way, the Farrell essay, just to be clear. It's like I don't think he's necessarily saying like, "Here's what we should do," you know? I think he's saying, "This will change. This will change, and this, and this is how we should think about this change. It's gonna be a cultural battle using, a new production technology." And he brings up this case of content producers who might now have cheap substitutes for their production methods and maybe, certain high-status content producers will lose their influence. You know, natural to think about, writers and journalists, for example, but maybe, we can think about content producers as being politicians. They're producing video content that is consumed-
44:44Seth: They're political. They're political.
Andrey: Or even lawyers who are, producing the text involved in lawsuits, right?
Seth: Well put. I guess, let me, let me kind of see what... Here's a sentence that I take away as sort of a thesis for the whole essay, which is, "We do not, as a discipline talking about political science, regularly think hard about how technology affects the deep structures of the political economy, government in democratic and autocratic regimes, and democratic representation and feedback. That has to change." Does he want political science to be political economy? I don't think that's what he's saying. I think he's saying something like maybe that this positive turn is too narrow. Maybe it's, maybe that's what he's saying, or what you started with, Andrey, that- ... We're, that right now, poli sci is asking too narrow questions relative to how big this change that's coming is.
Andrey: Yeah. I mean, I think that's what he's asking. Or that's what he's saying, I guess. But I don't think he points us very far past just the fact that this is gonna be this cultural battle, which, it's good because, we can speculate about what that means ourselves. Maybe that's the point.
Seth: Okay. So let's do a little bit of speculating about what does political cultural battle look like in the age of AI. To wildly speculate here, remember that show, Crossfire, where Tucker Carlson would wear a bow tie and people would yell at him? Okay. What if that, but we had AI agents yell at each other, and then somehow democracy? I mean- [laughs]
Andrey: I don't know if that would be engaging content, Seth. I.. I, what I, what I more, I can imagine, although I don't, I don't know how widespread this way, of working will be. I had a discussion with a friend, about a contentious political issue that I didn't know very much about, and what I did was I went to an AI agent, and I asked it to do deep research and summarize the evidence and so on, and it did. And you might think that even if it summarizes the evidence, that's not gonna be a values-free summary of the evidence. Even the fact that some things, as we've discussed many times on the show, some things are easy to measure, some things are hard to measure. And so the very fact that deep research results in certain types of artifacts, maybe tables with numbers in them from specific studies. Who knows how good they are? You know, that might be a persuasive form for certain policy discussions. Now, I'm not gonna lean into that too much 'cause I generally don't think that policy is about effectiveness. So, when it comes to political beliefs, it's really about, my side versus their side. I don't know. But that, that's one world we can imagine, that, you know...
47:56Seth: Well, but maybe that's the positive vision. The positive vision is, as we get more detailed possibilities in these debates, it becomes less red team, blue team, and it becomes more of, everyone puts their dot on the political compass, and then at the end of the year, we put a circle, that tries to cover as many of them as possible, right?
Andrey: Yeah. Yeah. Yeah. But that, that's kind of the most kind of, yeah, benign, maybe even interesting, but not the least radical of the visions that we might imagine. The other version of it is the opposite. It's the reverse, right? You know, someone-... Has a political position, they wanna support. And then they ask the AI to come up with the best version of the argument for this, and then they feel really good about themselves, and then they go about spreading that throughout this, the ecosystem, the cultural ecosystem, right? And, obviously this becomes even worse if there's, conflict of interest, and there always is. Right? You might imagine that, someone's beliefs might align with their, financial interests or other sorts of interests and, you might, you might have that. And then, and then, and then kind of naturally the question becomes, does how content gets distributed also get affected through this, system?
Seth: Right. It must, right? We'll all have our personalized AI agents who aren't just curating our Facebook stream, but all of the information- ... That's coming in at us, right? I would say that kind of at the most meta level, I'm optimistic that more communication, people being able to communicate to each other more robustly and more clearly, on net, once we've done the general equilibrium settling and people have adjusted to how newly persuasive everybody else is. It seems like that's-- we like to think that that produces good outcomes in the legal system, right? We like to think that, if both sides' lawyers spend a million dollars, we're gonna have a more accurate outcome of the case than if both sides' lawyers spend $1, right? That's kind of where, what I think. I don't, but I don't know. I don't know where it's gonna lead us. Do you, do you accept that premise? [chuckles]
Andrey: That's, that is very Pollyannish, if you will. Yeah, I think it's a possibility. I think it's-- I think I think weirder things will happen, but I have a hard time predicting it.
Seth: I think what I wanna read next after reading this is actually read some Glenn Weyl-style futurism and actually see, okay, this is how they think the future of voting is gonna work.
51:02Andrey: Yeah. The-
Seth: And then we can have more specific takes
Andrey: ... We're talking about, the Plurality Institute, right, and things like this.
Seth: Right. So yeah. So Glenn Weyl is one of many people kind of working in the direction of sort of digital democracy. How can we get kind of both more direct democracy and more technocratic rule through smart applications of these technologies? I know other people are thinking about these questions. I know, the Bostrom Future of Humanity Institute people were always interested in these questions. This essay, unfortunately, I think ends itself right when the conversation starts getting interesting. Right? It ends right before that.
Andrey: Yeah. But, which is, it's good, because, there's a lot more for us to talk about and explore. So yeah, maybe we can, we can kind of wrap this up. So after, reading and this discussion, kind of are any of your priors updated here?
Seth: Right. So I started by saying that I thought that AI will create new forms of governance, that we will need a new form of political science that focuses on this 60%. I come away reading this essay and, feeling that number has to come up a lot because [chuckles] it sounds like political scientists have not thought about this at all. And I was totally happy to say, "Did the cyberneticists have opinions about this?" But apparently, not enough worth citing [chuckles] or citing heavily. And so, I'm gonna go for maybe 60% on that to 90% on that.
Andrey: But okay. But what about kind of AI will lead to forms of governments that are so different from the current ones that we'll need a new field of political science for it?
Seth: Yeah. I'm up from 60 to 90.
Andrey: That's-
Seth: Right? You know, I-
Andrey: That's kind of what you mean. That's what you mean. Okay.
Seth: Yeah. That's what I meant to say.
Andrey: Yeah. Um-
Seth: On this-- Go ahead.
Andrey: Yeah. I guess on, for me, I had no prior on this, so I guess I shouldn't have a posterior. [chuckles]
Seth: No, I mean, well, I mean, you're, there's, you're allowed to have the prior of exactly 50/50 if that's the one you want to attempt.
Andrey: Well, I just, I, it's hard for me to just even conceptualize that, that prior. I-
Seth: Ambiguous. The answer is you cannot represent your beliefs with a number between zero and one.
Andrey: I don't know how to represent my beliefs here. I guess, I guess if we think, is AI gonna create transformative changes in governance that are gonna be hard to recognize from our current lens, eventually with artificial super intelligence, necessarily so. Um- ... In the intermediate worlds, I think we can fit a lot into our current frameworks. Yeah.
Seth: Right. For, yes, exactly. And I think that that kind of follows the structure of this essay, where we spend kind of most of the time on AI as governance tool, where it's, the politically powerful people are gonna do this using AI now, versus AI as governance, which is a lot more speculative at this point. Now the second question I thought was kind of a lot more interesting to sort of engage with in a quantitative way, which was something like a useful way to think about the political effects of AI is as a new cultural technology for us to use in battles in political influence. I started there at 40%, I guess 'cause I was mostly thinking about AI as reinforcement learning, and we can talk about that as kind of one of the limitations of this, essay is it really it seems like it wants to do a hard distinguishing AI as optimizer reinforcement learning tool versus AI as LLM cultural product creator. And it doesn't do a lot of...
I don't know. It kind of goes back and forth. It doesn't do, either do a distinction or do an integration. Setting that aside, do I think that that latter form of AI as LLM, AI as thing that can be persuasive or produce cultural artifacts, is that the right way to think? Is it, A useful way to think about AI? Oh, definitely go up from 40% to, 80, 90%. I think that is actually a pretty useful framing. The most useful way to think about AI, no. Probably the direct effects on economic outcomes will be more... Like, if you double everyone's income, you also [laughs] change politics, right?
55:39Andrey: Yeah. Well, I mean, yeah. And it, and it's not just that. If you just give the government a new technology to use, and don't increase GDP, that still has pretty profound effects, and that's not through the cultural sphere. I- ... But I agree. I thought a lot more about this and, I updated, I think, my prior to 15% from my 10%, that it's kind of a, the primary way to think about it. But it's a useful frame. Even though I agree with you it's not the primary frame, that I would still use for this issue.
Seth: Right. It's.. Right. It's... I read a sci-fi novel once which was about... And there's an aside in it where there was a battle between these two AI systems, and the two AI systems kind of put out, different little drones that fought each other. And there was kind of like this escalating arms race of, okay, well, he used this kind of drone, so I'm gonna use the counter drone. If you think about, if, kids at home- ... If you've ever played, an auto battler video game, that's kind of how this was described in the novel. And I'm just imagining what the version of that is in the cultural domain, right? Is that I've got my AI reading and writing to Twitter, you've got your AI reading and writing to Twitter, and we're gonna have this automated cultural battle where, I'm creating this whole AstroTurf society of people who want a lower marginal tax rate on sand, and you've got this whole society of people that wants a higher marginal tax rate on sand, and we're just having this kind of cultural battle in public that's 95% automated. [laughs] That's a possible future.
Andrey: It is. So most of the content, as Tyler Cowen likes to say, will be consumed by the AIs. That's another way to think about it, is that the training data, right, itself is gonna be AI generated.
Seth: Right. And now you're worried about model collapse. There's an aside here in this essay about model collapse. I don't think Farrell has anything particularly interesting to say about it other than that bad, baby.
Andrey: [laughs] all right. Well, on this note, shall we wrap it up?
Seth: I think so. I think we have justified our posteriors. And so, to our listeners at home, I hope that if you still consume any human-created content, it includes Justified Posteriors. And, I guess from us to you in Podcastville, keep your posteriors justified.
Andrey: Yeah. Peace out. All right, uh-