Robin Hanson: Radical Fixes for Falling Fertility and AI Alignment
Cold Open 0:00
0:00Robin: I realized not only is this humanity's superpower, we broke it, and there'll be hell to pay for breaking it. One obvious solution is just unleash capitalism everywhere. You let capitalists own governments, run them, you let them run orphanages where they sell the kids when they're grown. So far, this is what we're doing for AI.
Andrey: I have a lot of these deep uncertainties about what's gonna happen. But one thing I'd love is a market-based mechanism to allow me to ground my beliefs.
Robin: This is one of the ways in which prediction markets excel compared to other information institutions. It turns out to be very hard to manipulate, and that's a common problem in podcasts and journalism.
Andrey: No. Never. Not for us.
Seth: Not on this podcast.
0:37Seth: Welcome to Justified Posteriors. Today, we have joining us the legendary Robin Hanson, economist at George Mason and author of the Overcoming Bias blog, a leader in the development of a diverse array of chewy and influential ideas from prediction markets to grabby aliens. I'm Seth Benzell, always excited to evolve culturally alongside my audience, coming to you from Chapman University in sunny Southern California.
Andrey: And I'm Andrey Fradkin, coming to you from San Francisco, California, and we are so excited to have you on the show, Robin.
Robin: Great to be here. I have ex-colleagues from Mason who are now at Chapman.
Seth: Yeah, that's true. We got your experimental econ department 20 years ago, and we've been running on that Vernon Smith energy ever since. 100th birthday party coming up soon.
AI and Growth: An Internet-Sized Boom 1:32
1:32Andrey: So Robin, we're mostly an economics of AI podcast, and one thing we were curious about is whether you have any predictions about what GDP growth is likely to be over the next ten years.
Robin: I have a median estimate. There's a bit of variance, but at least I can put you roughly in the median. I'm looking at a burst roughly like the internet boom over the next ten years. I'd say the place we've seen the clearest traction is in software. Yes, there is math, but math is a tiny industry compared to software. Software is big. And software is an industry with elastic demand. A lot of people have wanted a lot more software than they've had because the price is high.
So I predict the software industry roughly triples in the next ten years. Twenty percent of that new revenue goes to AI. The rest goes to top software engineers, and that's something people should wanna be: the people who can manage the AIs. Not all of them can, but those who can will be getting most of that revenue. Roughly half of AI industry revenue will be coming from the software industry, and that would be roughly an internet-scale boom. So not substantially changing the overall growth rates, but supporting the continued growth rate. The world economy doubles roughly every 20 years, and that means on average every industry has to get about twice as efficient every 20 years. That's a really big demand, and so this is the sort of thing that could keep us on track with that pretty demanding requirement to double productivity every 20 years.
3:10Andrey: So it sounds like you're relatively pessimistic on the real physical aspect of AI. So robotics, perhaps biomedical innovation. Do you have a favorite set of bottlenecks that you think are gonna prevent us from achieving additional growth through that?
Robin: I'm more just going on priors. For a long time, every 30 years or so, we have some new burst of innovation in automation and AI, and typically they find traction in some areas, but not in most. It's really hard to overall improve a wide range of things. This new innovation is powerful, and when I see a range of applications that it works on, I go, "Great. That's a winner." But I don't presume that because you're able to do one thing, you can do everything. That's just way too hard. So the prior here is most technologies are limited. This technology will be limited, so far. So far it seems to be.
Andrey: Just trying to take the position of the people who are maybe more AGI-pilled on the economic effects. I think they might say, "Well, everything is bottlenecked by intelligence," and what they've created is not an automated software engineer per se, but a superintelligent system that can do intellectual tasks on the scale of the hardest things that humans have ever done.
Robin: The key metric is the economy. It's not about passing tests or impressing college professors. The key thing is the whole economy and the jobs that are being done and being able to do those jobs. In order to produce economic growth, you have to change a lot of jobs and make them be done better. All these different jobs take many inputs, including various mental skills, but there isn't just a one-dimensional intelligence. There are lots of different kinds of mental skills, and when you improve some, you don't improve all. You improve the ones you've improved, and the question is which ones did you improve?
Here we're improving a substantial number of skills, enough that you could have a big impact on something like software, and that's a pretty big deal. But they're not improving all mental skills. They're improving a particular set of mental skills. Now, one of the things is they write essays, and many essay writers tend to think, well, if you can write essays, you could do anything because, well, we essay writers could do everything.
Seth: Yeah, economics blogger is the final job.
Robin: Right. But it's just not true that if you're a good essay writer you could do any job. It's just not.
5:51Andrey: But it seems like we have a technology that allows us to pick a target, a certain set of skills, and then optimize our AI systems to do those.
Robin: Well, we've had more success with some than others, right? We haven't been able to take any task and do an optimization and make it happen. People have been picking the tasks they think look like winners that they're gonna be able to optimize and be impressive with. They haven't been picking tasks at random to improve, and many of the tasks they've tried to improve, they haven't been able to improve that much. So there is, so far, a limitation.
Andrey: I'm not sure which tasks they haven't been able to improve after choosing to target them. Do you have some in mind?
Seth: Once they have a good benchmark.
Andrey: No, no, no. I disagree. You create benchmarks of the things you wanna improve, and so it seems like anything they point at can be improved.
Robin: But enough to be able to replace the regular people doing it? That's the standard. Improving over previous systems, impressive, but the standard is replacing individual humans who are doing these tasks. That's the relevant standard for impact on the economy.
Seth: But is the relevant standard replacing individuals or replacing the entire system? You might imagine a universe where AI gets really good at all the mental skills except for cooperating. But it doesn't matter, 'cause the AI just needs to cooperate with the other AIs.
7:11Robin: Or cooperating with humans, which you might say is a different skill. But as you probably know, there's this thing called general-purpose technologies, a category in economics whereby people have studied past big transitions like the steam engine, electricity, the personal computer. And there's a standard result: even though these technologies are quite general and applicable across a pretty wide range of tasks, it often took several decades before they actually managed to get traction on productivity. The standard story is that swapping them in for the prior thing that did roughly the same thing didn't do that much. You had to reorganize things in order to get the big productivity gains, and it takes a while to reorganize.
Seth: So adjustment costs.
Robin: If you give me this 10-year timescale, I'm gonna say, "Well, that's before the major reorganizing timescale, so yeah, you're gonna see limited impact."
Seth: So if you're gonna point towards adjustment costs as maybe the biggest bottleneck, you might think of that in two different bins. There's the intellectual labor of trying to figure out the new way to do my business, and then there's the financial resources: I've got to go raise the billion dollars to build the new version of the company. It feels like both of those should be faster, right? We should be able to do the intellectual labor faster with the AI tools, and finance seems to be flowing very fast, so why shouldn't adjustment costs become less of a barrier?
8:33Robin: Innovation is a combination of invention and diffusion, and people neglect the diffusion part. But diffusion is actually the most important part. People focus on doing it the new way the first time, but the harder problem is to spread the new way to lots of other people, and that often requires somebody being pretty close, looking at what you're doing and saying, "That's pretty close to me. I wonder if the way they're doing that could be applied here." And that just takes a long time to spread. That's another big piece of inertia: the spreading of better practice, because every time somebody else copies a better practice, they have to adapt it a bit to their different circumstance, and that takes some intelligence and some experimentation. The first place that had the new electric motors in their factories and managed to make it work, that's just the first one. Now some other factory has to visit and go, "Wow, that seems to be working for you. Let's try that over here." And that's actually a bigger, more important part of the process than the first guy doing it.
Seth: Or they need to be killed by competition, and that can take years and years.
Robin: Right. Exactly.
Long-Run Prediction Markets 9:40
9:40Andrey: I have a lot of these deep uncertainties, just as you do, about what's gonna happen, although maybe I'm a little bit closer to thinking AI will have a slightly bigger economic impact relative to previous GPTs. But one thing I'd love is a market-based mechanism to allow me to ground my beliefs.
Seth: Why can't society just tell us?
Andrey: One of the things that I've been working on and curious about is the performance of prediction markets over a longer time horizon. We now have pretty good evidence that these markets are pretty well calibrated for contracts within a year or so of expiration. But some of these outcomes that we really care about are maybe five, 10 years out. What are your thoughts about long-run prediction markets and what it would take for them to be successful?
10:30Robin: The most clear thing we can say is that when we have a prediction market and some estimate we're trying to get, we're asking people to back it with capital. We say, "You have some insight. We've got a place to buy insight, but we don't buy insight from you unless you back your insight with capital." That's how a prediction market works. You can't just have a story. If you don't have capital, you have to partner with somebody with capital, because you have to make a bet in the market using your capital, and then if you're right, you get the payoff from your information. So that's a key feature of these markets: they require access to capital.
Now, you can have pretty thin markets with low capital and still be pretty informative. That's great. But if we want you to put a fair bit of work into it, then we need to pay you a lot, and that needs a thick market. And the thick market means you'll have to have a lot of capital to go up. So the more we're paying you for your information, the more access to capital you need to back that. Now, if it's a very short-term topic, you don't need to tie the capital up for very long. You just need access to maybe a lot of capital, but only for short periods, and you could repeatedly use that capital every day on a new topic and do 365 different topics in a year with the same capital, if they're all one-day topics. But the longer you need to hold onto your assets to get your payoff, the longer you have to put that capital in, and the opportunity cost of capital becomes more of a constraint on your participation. So that's the basic fact about these things: it's just cheaper to elicit information about short-term topics relative to long-term ones, because the long-term ones require people to tie up more capital over the longer run. It doesn't mean you can't do longer-term topics. It just means you'll have to pony up more payment to do that.
12:20It's also a matter that you don't necessarily need people to invest in the long run for a long-term topic. Most of the stock market is implicitly a long-term thing, but most people trading in the stock market are making relatively short-term trades. So what it's really about is: how long does your information last? Every piece of information has a certain timescale on which somebody else is gonna find out. If you wait that long to trade, you're risking somebody else trading on the information before you. So you need to trade on it before they do. It's also generally true that if you have a piece of information, you make more money on it by spreading the trade out over the maximum period you can. The shorter a time period you try to trade on, the less money you make. That's just a basic fact of the way financial markets work. So you wanna spread it out up until the point where somebody else is gonna learn that information. All information has a scale like that, and that's a key metric for trading. It's not about how long the fundamental resolution takes to happen. It's about how long each piece of information lasts.
For example, some people have high-frequency trading information. They just get fast information about some markets and trade it in other markets. That information is really short duration, so it's very profitable with respect to price, but the time period is so short that they can't spread their trades out. They have to make their trades really fast, so they can't make as much money on those trades. And it's not so clear that the rest of us really care about them profiting from that. But when we're designing these things, we wanna ask what kind of information we really want, and set up the system to encourage that kind of information. So again, to your question about long-term topics: it's not so much about the long-term topic. It's about the typical duration of the information someone could have on that topic. If you could have short-term information on a long-term topic, you can get people to reveal it with relatively cheap incentives, because they're gonna buy today, sell tomorrow to somebody else who buys and sells, and eventually the information gets out. Each trader didn't have to hold it very long.
14:29Andrey: My sense is it's more of an intermediate information duration for some of these. Let's say unemployment five years, 10 years from now. You're getting some information today, and there are states of the world where unemployment starts changing a lot in the coming year, and then you can get out of your trade. But there are states of the world where it might take longer to resolve some uncertainty. I'm curious, do you have a preferred way of subsidizing prediction markets? Let's say you want to get more people to trade on it.
Robin: A simple approach that we thought of a while ago is nice for a lot of cases. Say you had a market and you just paid per trade. That wouldn't be so good, 'cause people would just make a lot of trades that weren't very informative. You only wanna pay per information in the trade. So subsidizing a market maker is a nice way to pay for information, because only if you have information will you wanna trade against that market maker and move its price. However, it's not very discriminating in terms of time or price. If you just have a market maker that works at a wide range of prices and is there constantly in time, it's subsidizing information anywhere in the price range and at any time.
Seth: A market maker would buy and sell at the current middle of the bid/ask spread? What would they be doing?
Robin: A market maker would, for example, just have a current bid/ask, and then if you bought, the bid/ask would go up. If you sold, the bid/ask goes down. So it manages its bid/ask in response to the quantity of trades it receives. There's a certain thickness in terms of how far the price moves as a function of a certain quantity of trade. Relative to that, if you have a wider spread, it will be profiting as things move. You narrow the spread, and it moves to a point where it's just breaking even. And if you narrow it further, now it loses as things move, and that's how it's subsidizing the traders. If we want it to lose money, we can give it a narrower bid/ask spread, in which case that extra profit only goes to people who move the price on net, which is therefore a subsidy for people who have information.
But again, the problem is it's not very discriminating about where in the price range you're subsidizing, nor where in the time range. A key fact about information is that it's only really valuable for advising decisions. When you have decisions, you typically have a point in time when you expect to make that decision, at least some range, and you have thresholds that matter for your decision. So what you really wanna do is subsidize near your threshold in price and near your decision time in time. Now you're concentrating your incentives near where you actually wanted to know something.
Seth: You wanna buy the valuable information.
Robin: Right. Exactly. But people haven't been doing much of that. People have just been subsidizing with a simple market maker, and that's still a big win over what people have done previously. So I'm still happy that people do that. I'm just pointing to where in the future you could do better.
Andrey: And how would you subsidize for a specific time horizon or range of information?
Robin: The market maker's thickness can just vary with price and with time. So you just crank up the thickness to be strong near your thresholds in time and price and weak elsewhere.
17:39Andrey: I see what you mean. One other thing related to this, and I think Bob Shiller is associated with these sorts of proposals, is that these long-run contracts are useful for hedging. Let's say I'm employed in the software industry and I'm worried about my unemployment loss from AI technologies. In principle, I should be buying insurance against that sort of risk. Why do you think these sorts of markets haven't taken off? And do you think that now, with the latest spread of your Kalshis and Polymarkets, this will happen?
Robin: Years ago, the standard financial market regulators said explicitly that they didn't want to allow prediction markets because they were too small to allow much hedging, and the point of the markets was hedging. Their argument was there's not much point in small trades for hedging. You need to hedge with large trades corresponding to the level of your risk, and large organizations needed to make large trades. So in some sense, Kalshi is proving that small markets can find customers, and now they're trying to prove that there can be small risk hedging with these small markets. I don't know if they'll succeed or not. If they do, they will have disproven these previous regulators who claimed that you could only hedge risks in big markets.
But my interest hasn't been in a larger range of hedging. My interest has been in information aggregation. My argument was that there are other purposes of markets besides risk hedging, and that is actually more valuable. In financial markets so far, the main customer has been the trader, and the trader has had two desires. One is to hedge risk, and the other is to have fun, get action, entertainment: speculate. The size of financial markets so far has been dominated by the size of the demand for those two functions. Clearly, we're seeing that there's a larger potential demand than had previously been tapped for the entertainment part. That's what these new prediction markets are showing. But I've always wanted this new customer: the person who wants information and is willing to subsidize and create markets on their topic to get information. I think there's a much larger demand for that in the long run than for those other two kinds of demand. But that requires a new customer. Instead of the customer being the trader, the customer is now somebody who has a decision and wants information on that decision. We're starting to see some of that now, but it's still really small, and that's where I would like things to go.
Decision Markets and Why Manipulators Are Sheep 20:15
20:15Andrey: And where are we starting to see that?
Robin: Well, glad you asked. There's this company called MetaDAO in crypto, which creates what I call decision markets for governance. Many crypto companies, really all of them, have a token, and the token has a price in a market. Then a key governance decision comes before the organization, and the way to advise and even oversee that decision is a yes-or-no proposal. We make a token price if yes and a token price if no, and we approve the proposal if the first price is higher than the second, but not otherwise. This is what I call a decision market. MetaDAO made its constitution say it has to make its key decisions this way, and it's now had twenty-five-plus other firms who have incorporated under this sort of constitution, collectively making one hundred and fifty decisions this way. So they are now in the practice of using decision markets to make key governance choices in these crypto firms. That's the demand for information. These companies say, "We can't just rug-pull you. We can't just do whatever we want with the money. For key decisions, we need to get market approval through this mechanism." And that's how these companies are subsidizing these markets, so that the traders will give them information for making key governance choices.
Andrey: Just so I understand a little more clearly: let's say I wanna participate in this market. I'm gonna buy a bunch of tokens.
Robin: You could buy or sell, depending on which side you wanna go on.
Andrey: But let's say I want the decision to be made. I'm gonna buy a bunch of—
Robin: You want yes, okay? You could do that two ways. You could either push up the yes price or push down the no price. To push up the yes price, you just need cash: you go into the yes market and you buy tokens for cash. But if you think the yes is fine but the no is too high, and you wanna push down no, now you need tokens to sell in the no market, and you sell them for cash there.
22:18Andrey: The obvious thought here is that if I'm a whale, I could just buy the decision.
Robin: You think you could, but you can't.
Andrey: And what is the key thing that's preventing that?
Robin: All information institutions, including prediction markets and journalism and gossip and academic journals, and even podcasts, have a number of ways they could go wrong, or worries people have, and one of those is manipulation: people distort the output by pushing in distorted inputs. And that is a common problem in podcasts and journalism—
Andrey: No. Never. Not for us.
Robin: Not on this podcast. And gossip. But it's much less of a problem in prediction markets. This is one of the ways in which prediction markets excel compared to other information institutions. It turns out to be very hard to manipulate them.
So here's the simple theory. A simple model of financial markets, including prediction markets, is that there are two kinds of traders. I will call them sheep and wolves. The wolves are traders who have information, and their intent is to trade in the market on their information and profit from their trades. To do that, they need to trade against a sheep. If two wolves trade against each other, they've got opposite information, and on average that cancels. But if a wolf trades against a sheep, they make money, because a sheep is defined as somebody who doesn't have information. They're trading for any reason other than their information. And there are many reasons to trade. You could just be an idiot. You could have hedging needs, whatever it is. So sheep are traders who trade for reasons other than having information. Wolves are ones who trade on information.
The basic fact about markets is that sheep show up for whatever reason, and then wolves are attracted to the sheep. Wolves wanna go where the sheep are, 'cause that's how they make their profit. So the more sheep, the more wolves, and the harder wolves work to learn information and trade against the sheep. This is why, for example, a market on Google has more accurate prices than a penny stock: there are more sheep trading Google. Same for presidential markets being more accurate than a senatorial or mayoral race, because there are more sheep in the presidential market. So there are more wolves, more accuracy.
What does this have to do with manipulation? Manipulators are sheep. That's all you have to know. Manipulators are there for a reason other than the information they have, which means wolves would like to trade against them. And the more wolves expect manipulators to show up, the more they're gonna wanna show up on the opposite side of their trades, because they want to trade against sheep, and manipulators are sheep. This is not just some cute little story. We have game theory and math models that work this out. We have lab experiments. We have field data. This is actually how it works in the world. In markets, the wolves are attracted to the sheep.
25:07Now, there's actually a stark lesson here. Most financial markets work this way. The main reason most financial markets have the accuracy they do is that the wolves are feeding off the sheep. They all work because there are all these sheep willing to lose. Many of those are people who need to make trades. If you're saving for retirement, you have to put money in the stock market every year, and later you have to take it out in order to retire. Those are trades you can't escape having to make, even though you don't have information. But most of the traders in the markets are actually not those people. They're speculators. They're losers. They're fools, even. And most financial markets are basically run by taxing those fools. That's where most of the accuracy and most of the liquidity comes from: we tax the fools willing to show up in these markets, and that's also how prediction markets work. In some sense, we as a society have chosen that route to finance these markets. We could just directly subsidize these markets to get the information, if we were willing to pony that up. Instead we go, "Hey, these fools are showing up willing to be taxed. Fine, let's tax them." That's what we're doing.
26:11Seth: This is maybe a small tangent, but I can think of a third animal in this taxonomy, which maybe would be the ape. There's this theory that maybe the wolves have some good insider information, but they've committed themselves to some short-term price, and then all of the apes can get together and short-squeeze the wolves even though they don't have information. Aren't there market strategies where you can make money through coordination without insider information?
Robin: That's rare, but sometimes there are such strategies. But the fact that they're available to everybody and lots of people know about them means there's a game-theoretic equilibrium, a mixed strategy of who tries them when, and that equilibrium pushes down the profits you can make to barely cover your expenses. There's enough competition for that role that you don't have any exclusive access to it. Lots of people are thinking all the time, "Could I do the squeeze thing? Am I gonna profit?" And most of the time they fail. Sometimes they succeed. On average, there's no exceptional profit from trying those strategies.
Seth: Because the wolves would do that if it was better than doing their wolf thing.
Robin: Yeah.
AI Forecasters and Prediction Markets 27:20
27:20Andrey: So there's been a lot of hype recently over these AI forecasting systems, and I'm curious how you think they will affect the feasibility of prediction markets. Are they a substitute? Are they a complement?
Robin: The idea of an institution in economics is a structure that's pretty robust to the details of the problem it's tackling or the participants involved. You want a structure that will, in general, produce the kind of outcomes you want in a wide range of particular circumstances. That's what an institution is. A prediction market is intended to be such an institution. In particular, it's supposed to be a forum where different kinds of participants can use different methods and different sources of information, and the mechanism itself doesn't have to care. It just has this general approach of rewarding the ones who are accurate and punishing the ones who aren't, and that allows it to be agnostic about the race or the gender or the age of participants, or whether they're human or AI. As long as we have an API that machines can use, which we've had for decades now, because most financial market trading has been done by machines for decades, then we don't really have to care who's participating.
But if a new source of participants shows up that's actually more cost-effective, better able to sit in that role, the thing you'd expect to see is that it becomes cheaper to use this as a way to buy information. I've said what I want in the future is somebody who says, "I want to know about my decision here. I'm willing to subsidize markets on my decision to get this information." But you'll have to pay a cost, and that'll be the cost of the information providers. How much trouble is it for them to collect the information, aggregate it, and then trade on it? If a new population of traders shows up who is just better at that, you'll find that you don't have to pay as much to get the information you wanted. That's the hope you can have with AI: that they will just show up as a new, better participant who sits in the slot. We don't have to change the slot itself. We could just get better participants.
I still have to say that's mostly a hope so far, in the sense that financial market trading is one of the areas where, for a long time, a lot of people have consistently said, "Is AI good enough for that now?" And they throw AI at it, decade after decade, and I don't think they're getting that much traction even now. But that's certainly a reasonable hope.
29:51Andrey: I think of it a little bit differently. Let's say I'm a market maker. There's a fixed cost of me making a particular market. I don't know what a good baseline price is. Maybe if these AI systems are well calibrated and reasonably good, which it does seem from the recent evidence, it's a good baseline to start with. And then, yeah, the markets are still adversarial, and you don't expect—
Seth: You're not gonna make a lot of money trading only off of the AI prediction, because everyone else is using the AI prediction. But at least it should make it easier to market-make.
Robin: One issue is how far down the long tail of small markets we can go, and that's about the cost of starting up a new market, ending it fast, and having it be thin with not many participants. You might think AI would make the biggest difference there. Maybe you could make a new market with a new initial price, maybe you could settle it, and you wouldn't even need one hour of human time, because you'll have a bunch of AIs that participate very cheaply. So that does seem plausibly where you'd get the biggest gains. Unfortunately, so far the world just hasn't had lots of tiny markets because of regulatory barriers. We're having more small ones now, but still not lots of tiny ones. So that would be in some future where we allow lots of tiny markets and we have customers who find a reason for them. And then, sure, AI might be better placed to fill those tiny market roles.
AI Alignment: Prefer Law to Values 31:24
31:24Seth: I think one natural next question after thinking about AIs trading on prediction markets is asking about AIs and property rights: AIs as business owners and agents that can contract. You've said in a blog post that we should prefer law to values when it comes to getting AIs on our side.
Robin: That was a while ago.
Seth: That was a while ago. So maybe you can tell us your views today about the idea of both AIs today and near-future, human-level AIs being given property rights.
Robin: Worries about AI happen on two very different timescales. Some people worry about a very short-term, sudden burst in AI capabilities and sudden defiances or betrayals.
Seth: Pivotal actions.
Robin: Right. And then other people worry about what happens in the long run when AI is just generally better than humans at lots of things. Where do humans sit in that long run? There are worries about both of those, and I'm happy to talk about either one. But it sounds like you wanna talk about the first one.
Seth: Well, I'm talking about the next ten years: AIs that are reasonably extrapolated from the AIs we have now. Why would we want or not want to give them property rights?
32:41Robin: For a long time, people who worried about AIs worried about the betrayal AI: the AI who lies and cheats and tries to steal and basically tries to undermine the rest of our world and take over. And they've tended to focus on the solution of, let's make sure the AIs have the right values. If they have the right values, then they wouldn't wanna do this, and they would be nice to us. My point in this post from a long time ago is that this isn't how we make humans nice to each other. If we ask how humans are nice to each other, we use law and norms to make us be nice to each other. Yes, having appropriate matching values can help norms be obeyed and enforced, but more fundamentally, it's about having and enforcing the norms, and less about what somebody would value in the absence of such norms.
Seth: Is that right? I can imagine someone with values that are so alien to me that it would be hard to live in a neoliberal society with them. Their good is seeing me suffer. Can I cooperate with this guy?
Robin: So I guess you wanna think about what if we had a larger percentage of psychopaths.
Seth: Fair enough, yeah.
Robin: 'Cause psychopaths meet your definition. They feel the norms less emotionally, and they're more just obeying them because of enforcement. It might be true that if we had a larger fraction of psychopaths, we would have to step up our enforcement of laws and norms, but it's not that much harder to step them up. In a society of well-behaved people, you can relax how hard you enforce laws and norms and still have things work, and when people are worse behaved, you'll have to step up the strength of those things.
34:33Seth: I wanna push this a little bit harder. I wanna go from "they don't care about me and they're okay with me suffering" to "they actually wanna see me suffer." They actually hate me.
Robin: You understand that there are people like that right now.
Seth: Yeah, but—
Robin: They don't say that to you, but there are people around you in your world today who would in fact like to have you suffer. That's a common thing. This is the way humans are. Many people are spiteful, and they hold grudges. They have other sorts of people they resent and would like to take down a peg, and you probably fit the model of that for many people: the person they hate or resent, or someone like the person who treated them badly as a child. Yes, our world is full of people who would like to do that if not restrained by laws and norms, and it's laws and norms that are what make us mostly able to trust each other.
So if you asked, "Where could I retire? What country in the world could I go to retire safely?" you wouldn't wanna first look at opinion polls of what the values of people in that country are. What you'd wanna know is: what's the crime rate? What's the rate at which people actually successfully violate laws and norms? A high crime rate will make you concerned that you won't do well there, but as long as they are actually obeying laws and norms there, you don't much care why. Unless maybe you're imagining some revolution where all of a sudden a regime change would happen.
Seth: I can also imagine them being mean to me in ways that don't break the law.
Robin: Yeah, but that's happening now.
Seth: Sure.
Robin: People are in fact quite consistently mean to each other in ways that don't violate laws or norms. Have you been on social media?
Seth: I have been on social media. But I guess that's why, when I decide which country I want to retire to, of course I care very much about the murder rate, but I also care about surveys of "are people nice here?"
Robin: But "are people nice" is different from "what values do they have." That's the key point. Nice is about the behavior, as enforced by the usual levels of laws and norms.
Andrey: Famously, Southern hospitality. It's not necessarily indicative of their other actions.
36:43Robin: So the point I made in that old post is that that's what you should worry more about for the AIs. You should want to see that they're law-abiding. You should want to see that they have norms, that they understand them, that they see the incentives of norms and laws, and that they are in the habit of obeying because they realize those incentives. That's what you want to set up. It's more important to set that up than to decide their ultimate values.
Honestly, look, most people don't know their ultimate values. I think this is worth pointing out. We evolved a set of habits in different situations, and we have urges and heuristics that we use in different situations. That's what humans are: an amalgamation of various heuristics and desires in different contexts. But we never evolved some coherent overall structure that makes it all consistent. That's just not who humans are, and that's bothered a lot of philosophers for a long time. You sit down and say, "Yeah, but what overall set of values makes sense of all these behaviors?" And it's really hard to figure that out. Mostly, we never figure that out. That's the nature of humans: we are this somewhat random amalgam of habits and heuristics and desires that produce the world we're in. A lot of that is shaped by norms and laws and the habits of people around us, where our behavior matches theirs in ways that work out. But if you ask, "Well, what are your values behind that?" we don't know.
38:14Andrey: I agree with you on this point. One of the thoughts I had following on this is that the way AI models are currently trained is very much "let's find the ultimate set of values." Some of those might be meta-values; we've done a previous episode about the Anthropic Constitution. But I've always thought it's worth having the experiment of training the models to maximize reputation points, or to give it a finite budget—
Seth: Expected approbation of the crowd.
Andrey: Yeah. Give it penalties for breaking various laws, for example, and then see where that gets us. Using an analogous set of reasoning to yours.
Seth: Robin, I want to come back at this claim that values are so inconsistent. How could we put important decisions on something as nebulous as a human value? I got news for you: the law is pretty ambiguous and contradictory too. So I'm not sure that's such a strong defense of law over values.
Robin: What I meant is abstracted values, expressed at a high level of abstraction so as to be consistent across a wide scope. Clearly, in a standard decision-theory sense, we have utility functions. Any set of consistent actions has some set of probabilities and utility functions that explains it. The question is just how coherent that is at larger, abstract scales. Much of moral philosophy is trying to express pretty abstract principles that, if they were in fact our values and we applied them, would be relatively easy to express and consistent across pretty wide scopes, and that's the thing we don't find. But we do, of course, have values at the detailed level. We make choices, so yes, there are priorities.
40:06Andrey: But I do wonder, Robin, have you thought about the objectives AI models should be trained on? Have you read much about how they're currently being trained?
Robin: I have not been tracking the details. But a key question for a long time has been: when companies make AI models, and stuff happens in the world in response, and they track that, can they respond fast enough to do an adaptive evolution of the systems? For most technologies in the past, we mostly evolved them incrementally, adapting to their use and what happens with them, and that's mostly gone okay. There might be some exceptions where you say, "Oh, this adaptive response thing won't work because there's some unusual problem." And many people want to say that's what's happening with AI: with AI, you can't just trust this adaptive process to react to how things are going. You'll have to have some fundamental theory and anticipate a big jump in the future, because if you just go adapting, that'll go badly. I'm not very persuaded by that argument. It seems to me that we do want to mainly do the incremental adaptation thing, that we are successfully doing it, and that's what I'd suggest doing.
So I don't want to tell companies how they should do it. I want them to try a bunch of things and see which ones work out as they track. AI companies are tracking in quite fine detail, in time and by customer, what the customers are doing with their products and how that's going. That looks pretty good for adaptive response. The worst case of adaptive response is you send a product out and ten years later you get feedback. But here, we're getting feedback daily about how people are using these, and when things have gone wrong, they've been relatively minor. I gotta say, the strength of desire to regulate this industry with pretty draconian regulations is really out of proportion to the actual harms that we've seen from individuals using these products. By most historical standards, this is a pretty innocuous product as it's actually been used, and the rate at which anybody has any reasonable complaints about it is pretty small.
42:14Andrey: The Hugging Face incident, you're—
Robin: It's small.
Andrey: —not that big of a deal?
Robin: Right. We're just talking teenagers going on a joyride, doing some graffiti. It's the sort of thing that kids do sometimes. If it's not very damaging, then yeah, it's not a big deal.
Seth: Some AI people have concerns that we're gonna get used to using these AI systems, and there will be these longer-run repercussions. Although the arguments seem very similar to me to the arguments about social media.
Robin: Or television, or the novel, or the bicycle, or all the technologies we've ever had people complaining about.
Seth: I mean, the printing press did lead to several hundred years of religious war, but—
Robin: Indeed.
Seth: —it worked out in the long run, I guess.
Cultural Drift: "We Broke It" 42:59
42:59Seth: One topic you've been working on and blogging about more recently is this idea of cultural drift, or cultural maladaptation. Can you briefly remind the audience what you mean by cultural drift, and why it's bad?
Robin: As you know, I've been a pretty optimistic technologist and futurist for a long time, just projecting previous economic trends. And then I noticed we had this disturbing fertility trend, and I dug into that, and I realized that the proximate causes of this disturbing fertility trend were all cultural trends. And I said, "Why are all these cultural trends dumping on fertility?" So I switched to studying culture and figuring out how it works. And upon studying that, I realized not only is this humanity's superpower, the key engine that's distinguished us from all the other animals even more than our intelligence, but we broke it. And there'll be hell to pay for breaking it.
44:03This will need a little bit of explaining. First, I wanna make the distinction between "of" and "in." Let's think about DNA evolution as an analogy. If we think about species, there are some habitats that are fragmented and small, like rivers and rainforests and coral reefs, and then there are some large habitats, like chunks of the ocean. The large habitats have large species, and the small habitats have small species. In a large species, evolution in the species goes faster. There are more places for an innovation to show up, and then it can spread to the whole species. In a tiny species, evolution in the species is gonna be slower. But then there's evolution of the features that a species has in common. All the members of a species have to have enough shared mating structure that when they mate, new things come out that work. Those features are defined by the species, and to have evolution of those, you need many species. So we have better evolution of species in fragmented habitats like coral reefs, and better evolution in species in the oceans. And when we look back and ask, "Where did most life today come from?" more of it came from the fragmented habitats, suggesting evolution of species actually mattered more than evolution in species, which might be surprising.
45:12Okay, so that's an analogy. Now, cultural evolution is a lot like DNA evolution, but it's faster and better. It's also crude, because we're pioneering it. Humans started this, and it wasn't happening much before. With DNA evolution, you have two parents, the people who handed you your DNA. In culture, you have as many parents as there are people whose behavior you model your behavior on, people whose behavior you saw and are now being like. And you have as many children as the people whose behavior you influence. This influence through parents and children can happen fast. It doesn't have to wait until someone is born to do the next iteration, and it can spread fast around the world and jump very far. So cultural evolution is a powerful thing.
It also has "of" and "in." A culture is a group of people who share a bunch of norms, status markers, and values, such that if you were to visibly deviate from them in that culture, you would be punished. Just like with a species, the members of a culture have to have some things in common to function. So there's evolution in cultures and evolution of cultures. A few big cultures would make for great evolution in the culture of the things you're allowed to vary. But to have evolution of cultures, you need many cultures. That's the key problem.
46:37Now, natural selection, Darwinian evolution: we like to think of it as an innovation process, making things better. But its first task is to prevent stuff from getting worse. Its first task is to be a control process. And natural selection has parameters such that, if they go bad enough, it stops working, things get bad, and control fails. Fortunately that isn't the usual case, but it is a common case. Natural selection can just break and go bad.
So if we think about the evolution of cultures, I want you to think of this mathematical model of natural selection in general. There's a high-dimensional space with an adaptive region in it, a good place. And there's a cloud of points that need to track this adaptive region. The parameters of the process are: first, how many points there are. Second, when a point's in the middle of the region, it grows and splits, and outside the region, it shrinks and dies. That's the strength of selection pressure: the points need to feel a difference, and have a different effect, inside versus outside. Third, how fast the region moves out from under all the points. And fourth, how much the points wiggle wildly, independent of everything else. You can see that at one extreme, it'll do fine. The cloud will track the region. If we move all four of these parameters to the other extreme, it'll do badly: the cloud will fail to track the region, and the points will all just die. That's what happens in natural selection. I'm gonna apply that now to the evolution of cultures.
48:09Think about the last few centuries of cultural evolution. Three centuries ago, we had hundreds of thousands of peasant cultures around the globe, each relatively independent of its neighbors, basically running its own world. These peasant cultures were near the edge of survival. They faced strong selection pressures: famines, pandemics, and wars that would kill them off if they went bad. High variety, strong selection pressure. The world economy doubled roughly every thousand years, and that was the rate at which they needed to change to keep up with the changing context. Manageable. And they were very conservative, trying not to change. They didn't think they were changing. They thought they were staying the same. They did change, but not very perceptibly, so they weren't wiggling much.
Now, what's happened over the last three centuries? First, we merged peasant cultures into nations. We created national cultures out of peasant cultures, a great reduction in variety, and then we merged national cultures into a global monoculture, an even further reduction in variety, especially among elites. So variety is way down. Second, these few remaining cultures are not much being killed off by famines or wars or pandemics. We're rich, we're healthy, we're peaceful, so there's not much selection pressure on them. Third, the world economy doubles every 20 years instead of every thousand, a factor-of-50 increase in the rate at which adaptation needs to happen, because the world's changing that much faster. And finally, instead of being conservative and trying not to change cultures, we decided to make cultural activism our highest honor and to celebrate the people who go out of their way to change our key norms and values, our culture. So all four of these parameters have gone bad over the last few centuries.
The other prediction is that because we have fewer, bigger cultures, we have better innovation in culture, and so we are in fact having faster technological and economic innovation. The things that are allowed to vary inside cultures are going great. But partly that's what's causing the rapid change that cultures are finding it hard to adapt to fast enough.
50:20Seth: It seems like if I was integrating everyone into this one global culture and getting rid of our cultural Galápagos Islands, you would also get at least a one-time dividend from that.
Robin: We did, in fact. Yes. What happened is we merged cultures that were separate, and we didn't do that uniformly, symmetrically, or randomly. We tended to have people copy the more successful previous cultures, so in the process of merging all these little cultures into a big one, they were modeled more after whatever had been temporarily successful. That was a way in which adaptation was promoted. It doesn't last after you've done all the merging, but yes, over the last few centuries, we have been ahead in the game, not just from having big cultures with a lot of faster innovation in them, but because we picked the most successful ones, and others copied and modeled themselves after them.
Seth: Right. So we got a one-time dividend, but now we've set ourselves up to fail.
Robin: Unfortunately, right. The prediction here is that we are no longer tracking the adaptive region because we broke this key process, and I think the clearest sign is falling fertility. For a biological species to be shrinking in population in times of plenty and peace and health, something just has to be going pretty wrong.
Ought From Is, and Why Fertility? 51:36
51:36Seth: Let's talk about that for a second. I guess there are two questions I have here. The first is the ought-from-an-is issue: okay, sure, our culture is making us reproduce less, but how are we gonna get a normative conclusion from that?
Robin: The entire analysis I'm doing is from an evolutionary point of view. It's about what natural selection selects for and what fails to be selected for. That's not the same as what you like or what you think is good or moral. But everything you love and value was created in you by natural selection. There is no other source. You were created by natural selection, and all the things you love and you hate, all the good parts of you and all the bad parts of you, were given to you by natural selection. There is no other source.
Seth: I was causally determined by my past.
Robin: Yes, and this was the main process.
Seth: But I guess some of that was contingent, though.
Robin: Of course, there's some randomness. The consistent part was natural selection, and then there's a bunch of random stuff that natural selection would winnow out over time if it had a chance, but it hasn't had a chance yet.
Seth: Maybe that's the part I like, though.
Robin: It might well be, but that's the key point. Whatever you love, in a world of natural selection, of Darwinian evolution, nothing you love can last unless it's mixed in with an adaptive package. You could just say, "I don't care about the future. I am who I am, and I'm gonna live for today and enjoy my life and maybe my kids, and to hell with the future." In which case that attitude is going to be selected out, and the future will be populated by other things. If you aspire to influence the future, if you have things you love that you would like to persist into the future, now you face a difficult choice, because they don't automatically persist. They only persist if they're mixed into an adaptive package. So the question is, can you find a way to take some of the things you love and mix them in?
I would make the analogy: say you're in a city and there's an invading army, and you're told in the middle of the night you have one hour to pack a suitcase and leave. You can't take your whole house.
Seth: Pokémon cards.
Robin: Maybe, but the key point is you have to choose, and it's a wrenching choice if you have many things you love and you can only take one suitcase. We may have to face a choice like this. What do we wanna save and take into the future, and how much are we willing to pay to make that happen? Because by default, things don't last unless they're adaptive.
54:11Seth: Okay, that's a good answer, Robin. My next question would be: sure, I wanna be adaptive, 'cause I'd like the things that I like to be propagated into the future. But why does that have to look like fertility? Why couldn't it look like living longer, or creating artifacts that outlive me, or just making my GDP super big and having lots of copies of my favorite sentence written down?
Robin: The measure of adaptation is your net total influence on the future. So if you write lots of copies of your name down and that ends up having a big influence on the future, good for you. It doesn't look particularly likely in our world that simply writing down a lot of copies of your name will work, because people won't actually read all those copies. They'll read a few, and that won't influence them much. But yes, in a world of cultural evolution, it isn't just about who has your DNA. It's about all the different channels by which the future is influenced by what you do and who you are. The point is, in this theory, that's all going bad. The abstract story is that your habits are not evolving to achieve that. They are drifting away from achieving it, and falling fertility is a clear sign of that. The fertility channel is only one channel of influence, but it's a nice, clear sign that something's going bad. And plausibly, if you look at the other channels, you'll see that they're going bad too.
Three Proposals: Capitalism, "Number Go Up," and a Sacred Goal 55:35
55:35Andrey: So Robin, last time we talked about this, I think it was at a Manifest conference, it almost seemed like you were a bit nihilistic. Like, well, maybe the Amish are gonna keep on reproducing, and they'll take over the world. It's almost like we can't even really coordinate to change this trajectory. Or do you feel more optimistic these days? Do you have a proposal?
Robin: There are two parts to the analysis. One is to first diagnose what's really going on here, and then come up with what we could do that, if we did it, would plausibly solve the problem, and then sketch out what inputs that would require: what sort of resources and changes of mind would be required to implement those changes. I am happy to think that I now have a set of scenarios that I think would work. And then when I explore who's interested in them and who I might recruit, that's where I get discouraged and go, "Wow, people just don't seem very interested here." But I'm gonna keep trying, and I would love to be surprised and find a wider range of interest. I'm writing a book, and the hope is that somehow I could inspire more interest with the book than I am through conversations like the one we're having now. It's such a huge problem, and at the moment, I'm not seeing very many people worried about it, so I do feel an obligation to do everything I can to call attention to it and point out such solutions. I've explored a lot of possibilities, and some things that looked promising initially fell away, but I've got three proposals that I think still hold up as things that could work. The problem is they each are pretty big asks.
Andrey: All right, so what are they?
Proposal 1: Unleash Capitalism 57:17
57:17Robin: The first is: we can frame the problem as, we invented capitalism, we unleashed capitalism, and in the areas it's been allowed to run, we've had rapid, powerful innovation and evolution. But there are a bunch of areas of life where we did not let it run, including fertility and governance and some other key areas, even law. And in those areas, because we haven't unleashed capitalism, the evolutionary process is weak and falling way behind, and that's the key disconnect: capitalism rushing forward and the others falling behind, stumbling and falling down. In that framing, one obvious solution is just to unleash capitalism everywhere.
Andrey: Can you be a little more concrete about what you have in mind?
Robin: Yes. You let capitalists own governments, run them. You let them run orphanages where they sell the kids when they're grown. Whatever part of your society could be going wrong, you let capitalism come in and own all that stuff, hire people to make that stuff, sell that stuff, everything. And a lot of people hate this. Just from my description, you can see. But I think it would work. And I will say, so far this is what we're doing for AI. So far, we have not restrained capitalism in AI very much, so if we just keep doing that, I could have hopes that an AI-dominated world would be a world where capitalism runs most everything, and it would be solving this cultural problem that way.
58:48Andrey: One of the maladaptations might be that there's just a lower demand for children than we'd want as a society, and I don't know how capitalism would solve that if people don't want the children. Or do you have in mind that one very rich person wants a lot of children?
Robin: No, no, no.
Seth: Get rid of pensions.
Robin: No, no, no. You're thinking of childhood as a gift or something. The fundamental economic question is: do I value my life enough that I'm willing to pay for it? And if I do, there's a deal to be made between me and my parents. Even if they're completely selfish, I can pay them enough to have and raise me. That's the capitalist solution to fertility. We block that 'cause we want fertility to be a gift.
Seth: We have that right now, with Social Security. We tax the young and give it to the old.
Robin: Right. So we should just go further with that. The simple capitalist solution would be to let parents endow their kids with debt that they owe their parents, and then the parents could sell that debt. That's a simple capitalist solution, but everybody hates it. They don't wanna let parents do that. But this is what the government does already. The government endows every new citizen with their share of the national debt, and nobody minds that much.
Seth: Robin Hanson, pro Ponzi scheme.
Robin: Okay. But people hate the idea of parents endowing their kids with debt, so a simple near-solution is to say, "Okay, fine. Give parents a share of their kids' future tax revenue as a property right." Now you're doing the parents-endowing-kids-with-debt thing, but you're doing it through government, which people think is okay. And that would be a free-market, win-win solution. The parents get paid enough that they're willing to have the kids, the kids' lives are more valuable than what they have to pay for them, and everybody wins. That's the capitalist solution to fertility, but culture makes us not like it. There are roughly a half-dozen key trends that have caused fertility decline, and people love them. If we did this, giving parents a share of their kids' tax revenue, people would have to find a way to reverse these cultural trends in order to get that money, and people wouldn't like that, 'cause they love these cultural trends. That's my guess as to why people are just not very interested in the win-win solution.
Andrey: I see your solution, but if it was up to the parent and the child, I don't think there could be a credible commitment from the parents to extract from their child—
Robin: Law allows credible commitment. Full capitalism would allow those contracts to be created. You make the kid, you can endow them with whatever contracts you want, and we're off and running. But again, most people aren't gonna like this one.
Proposal 2: Govern by "Number Go Up" 1:01:19
1:01:19Seth: Okay, so that's option number one, hyper-capitalism. Let me hear option number two.
Robin: Number two: even with relatively small groups, we could commit to measuring their cultural success over the next century or two. That is, we could commit to measuring, in a century or two, all the behaviors and DNA that exist then and tracing their causal influence back to people and groups now, and therefore answer the question of who ended up having more influence. That would be a metric of actual adaptive success. Then we could make futures markets today on that eventual measure of success, and govern groups today by "number go up." We say, "You're the leader of our group. What we want you to do is make our group adaptively successful, and we're gonna empower you with resources and authority to make choices to do that. But the way we're gonna evaluate your performance is these futures markets estimating our future adaptive success." And we could do this with relatively small groups.
"Number go up" is my name for several different ways you could use a number to drive governance. One simple mechanism is that when the number goes up, you keep them, and when it goes down, you fire them. Another version is you give them a big financial payout in proportion to how much the number increases. And a third is to use my decision-market concepts: have betting markets decide particular decisions based on this metric. I, of course, prefer the last one, but all of them could be a way to govern a group by a number, and here the key number would be adaptive success.
Now, the major problem with this is that almost nobody cares much about adaptive success. It's not something people are very inclined to prioritize in evaluating or incentivizing their leaders. And social Darwinism, which is the name for trying to be successful as a group, has been given a terrible image. People say that's what the Nazis were trying to do, and you don't want to be a Nazi, do you? So then you don't want to do this. But this would work for whatever groups do it, even if it's only a small fraction of groups. The groups that decide, "We want to be adaptively successful," commit to measuring their success later. They have assets that pay off in proportion to that later, and they have futures market prices on those assets today. Speculators would adjust their estimates of that future success, interpreting policies today and guessing what will lead to success, and that would be the incentive for leaders to do the things that would actually be successful, at least in the eyes of the speculators judging it.
1:04:08Seth: And I guess you don't see an issue with potential overcorrection in that direction? When we started, we said, of course you want to balance adaptiveness with whatever other goals you have, but you could theoretically go too far.
Robin: I think they would just put some weight on adaptation, not full weight, and the question is how much weight would be enough. If you don't put enough, it probably would just fail. If you put 100%, it has a much better chance of succeeding. You might think, "I could get away with 98 or 95, maybe 90," and then there would be this chicken game of how much weight we really need to put on this to succeed. And we could actually have markets estimating the chance of success with given weights.
Proposal 3: A Sacred Goal 1:04:46
1:04:46Seth: All right, we gotta hear idea number three.
Robin: Okay, so idea number three is easier to sell, but it has a bigger constraint: I need a bigger group to apply it to. Number two works with smaller groups, but number three needs a group that's big enough that its actions will affect global trajectories. The United States might be big enough. China might be big enough. But maybe Mississippi isn't. For this group, we're gonna do the "number go up" thing, but with a number tracking a sacred goal that we can get behind and that happens to be inconsistent with civilizational collapse.
I did some polls on this. One example is: when do a million people live in space? A lot of people think, "Wow, I could get behind that," or, "I think many people could get behind a community that had this as their sacred goal and governed itself according to how successful it was at achieving it." It's about when this happens. The sooner the better. Now, the key point is we need a goal like this where, in order to succeed, you have to prevent civilization from collapsing. You have to have some big chunk of the world that doesn't collapse and fall away, because that would put off success to a date a long way away. A related goal is: when do we achieve physical immortality, like average lifespans of 1,000 years? That's another goal people could find inspiring, and it's also a goal where, if civilization falls in the next century or so, success gets put off for a long time.
The advantage here is that people can more emotionally get behind this goal. It's not abstract adaptation, which almost nobody feels very emotionally attached to. People do find these goals emotionally resonant and compelling. And once it's a sacred goal, you are ashamed to abandon it and proud to sacrifice for it.
1:06:30Seth: I like this angle, because one of your great blog post series is on what makes something sacred, which is such an interesting thought. And when I think about the arc of cultural history, or the long arc of justice, that's a key thing people would point to: "Oh, that's the sacred thing. I don't want you elites telling me how my culture should evolve." So I think it makes a lot of sense, instead of making this cultural-directedness agenda about destroying a sacred thing, to make it about adding a new sacred thing—
Robin: —orienting around it, discovering it. We never create the sacred thing. It finds us. It creates us. That's the norm of the sacred. But yes, we would realize that we had always wanted a million people to live in space, and that this was our destiny.
Seth: It was our destiny to always want that.
Robin: Yes. And that's how the sacred would go. On the one hand, this lets people feel more participation and choice about our agenda and our goals together, whereas adaptiveness feels like I'm just giving in to the universe and not setting my own priorities. And then the futarchy version would give wider participation. If we just give one leader a huge incentive, they might achieve it, but the rest of us are on the sidelines and don't feel so involved. With decision markets, a lot more of us could be directly involved if we so chose, and we might find it more meaningful to be involved more specifically in advising these choices.
Lightning Round: Sci-Fi Authors, Overrated or Underrated 1:08:08
1:08:08Andrey: I wanted to do a very specific overrated-or-underrated series focusing on science fiction authors, because I assume you've read a lot of science fiction.
Robin: I have read many over the years. The main problem is that I've become soured on science fiction the more I've learned about the world, because the more I learn about the world, the less realistic I realize they are. But of course, I have been inspired that they are trying to think about the future and envision things about it. That's where I sit mainly on over- or underrated. That's the main axis, and that's about my learning over time. Also, the world of science fiction today just has a very different vibe and aesthetic and priorities than science fiction fifty years ago, which is where most of the fiction that inspired me came from. Science fiction today isn't very interested in foreseeing the future or future trajectories. It's much more interested in the inner lives of the characters and how they're oppressed by other characters and that sort of stuff.
Andrey: Well, all right. Let's try it out.
Robin: Go for it.
Andrey: Vernor Vinge.
Robin: He would be hard to overrate. He's pretty high, so I would leave him up there. If other people don't appreciate him, then he's underrated. He's pretty high.
Andrey: Isaac Asimov.
Robin: He is really quite famous and well recognized, so I don't think he's overrated, but I also don't think he's underrated. He was probably correctly rated.
Seth: Never learned Foundation's tax policy.
Andrey: Yeah, I guess that's an interesting one, where he's inspired so many people, but at the same time, I don't think he gets the social science right.
Robin: But interestingly, he had the idea of social scientists trying to use social science to direct a culture. And that very idea is inspiring to people, even if his model of how culture works was bad and not very informative. The very idea of studying social science in order to direct the trajectory of a civilization is an inspiring concept. And I've asked LLMs what science fiction character I'm most like, and they'll give me Hari Seldon.
Andrey: That is an honor.
Robin: As somebody who's trying to direct a civilization by understanding culture.
1:10:13Andrey: All right. Neal Stephenson.
Robin: Probably roughly correctly rated. His books were fun. And he did explore different kinds of anarchic systems in some of his novels, and he correctly isn't at the scale of Vinge or Asimov.
Andrey: Okay. Ursula Le Guin.
Robin: Overrated, I guess. I read one of her books, about the revolutionary planet or whatever.
Seth: The Dispossessed?
Robin: Yeah, that's right.
Seth: Yeah, the anarcho-socialist one.
Robin: I read that in the last couple of years. I haven't read very many science fiction novels in the last few years, so she's one of the few I've read recently. I thought it was interesting, but also just so-so.
Andrey: Then one final one. If you had to recommend a science fiction author to our audience that we haven't asked you about yet, do you have one that you particularly like?
Robin: Not really, 'cause you've asked about many of the ones I would mention. I will say that my book, The Age of Em, is like science fiction.
Andrey: Of course. We have read it. Yes.
Robin: Except there's no plot, and there are no characters, and it all makes sense. And that last part is the hard part, having it all make sense. I really now value a very consistent, well-thought-through scenario. Most science fiction authors cut a lot of corners in order to have a plot and engaging characters, and they aren't prioritizing a completely consistent, thought-through world.
1:11:35Andrey: All right. Well, thanks so much. This was so much fun. I think our audience is gonna love this.
Robin: All right. Great to talk to you guys.
Andrey: Well, thanks for joining us for another episode of Justified Posteriors. As a reminder, we have written down all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out. And if you liked the episode, please share it with your friends.