Economist Tyler Cowen on How ChatGPT Is Changing Your Job - Ep. 7 with Tyler Cowen

If ChatGPT stopped existing today, how would that affect your productivity?

I would feel much less smart. AI is for learning. Perplexity is for references. Google is for links. I was in Tokyo. I use it as my universal translator. Paraguayan food. I was in Honduras. Mostly when I travel, it adds flavor to the trip itself.

I actually created a clone of you. I want to do a segment with you called the Tyler Test. I ask you a question, I'll have you answer it, and then I will also ask the clone that I made the same question. We'll see if it answers the question passionate in the way that you would. What are the core lessons of economics?

The first would be incentives matter. The second would be there's always an opportunity cost.

Tyler, welcome to the show.

Happy to be here. Thank you for having me on.

Of course. I'm really excited to do this. For people that don't know you, you're an economist at George Mason University. You're a prolific writer. You've written, I 17 books, and you've been writing your blog Marginal Revolution for almost, or over, 20 years. And I find you to be a just incredibly deep thinker. You think a lot about the impact of technology on life, work, and the economy, and I think it's all incredibly relevant to, to the show. And I'm just, yeah, super excited to have you.

Happy to be here. Great.

Cool. So what we tend to do on the show is talk very practically about how smart people use ChatGPT and AI in their lives to get work done. But I think before we get to that, there are some sort of high-level questions I want to ask you about your work and your view on AI and its impact on technology and jobs and the economy. So I think let's start there. And in particular, I think there are a lot of people right now who are trying to figure out what AI is and how it's going to affect their work lives and the economy more generally. And I think you've been thinking about that question about maybe AI in particular for a little while, but like generally about how technology affects our work lives for a really long time. Um, and in your book Average Is Over, which you wrote in 2013, you talked about this like stratification in the economy that's driven by what you call— what you called intelligent machines. And this is before like real AI had even sort of come out.

Um, well, in chess we had real AI in 2013. I guess that's true.

I guess, uh, uh, you know, before— let's say just before this generation of consumer AI had come out or was even like on the horizon. Um, and one of the things that you wrote is that if you and your skills are a complement to the computer, your wage and labor market prospects are likely to be cheery. Um, and if your skills do not complement the computer, you may want to address that mismatch. So you talked about this sort of stratification between, um, intelligent technical knowledge workers and the rest of the economy. Um, and, and, and you saw that gap and you saw that widening in 20— in 2013. And I'm just kind of curious for you to talk about prediction, like why you wrote that, what you saw then, and what you think now given the current generation of AI models.

Well, for a long time, even well before 2013, I thought artificial intelligence was the most likely place for the next major technological breakthrough to come. And I'm not an AI expert in any technical sense, but my intuition there was pretty simple. I was quite an avid chess player when I was very young, And at the time, people thought, well, AI can't really play chess. Chess is too conceptual, too complex, too non-legible. And pretty quickly, as I think you all know, AI managed to overcome all those hurdles, and it's now almost godlike when it plays chess.

Right.

So I thought, if AI can do that, chess is really hard. AI can do all these other things in time. And I was convinced this would happen.. And it's happening essentially. So that's the background of the book. And I wanted to be ahead of the trend and write about where I thought it was all going.

And, um, and I guess in, in your— so I think you're— I think it's clear that you were, you were ahead of the trend. Um, and, and I guess you, you made some predictions about, uh, what, how that would impact the economy. Like, do you feel like that was, that was spot on? Do you— are there any revisions that you want to make given what you I think it's probably at least somewhat surprising the exact specifics of this current generation of generative models. And I'm sort of curious how you reflect on that.

What's happening now is a work in progress. I don't feel that we know yet which groups of people it will help the most and harm the most. Like many people, I didn't expect that the AI breakthrough would have so much facility with words. And with emotions and with bedside manner. I don't mean that I ruled out the possibility, but back then I would have been surprised if I was told, well, those are some of the things that we'll be best at. So I thought it would be a bit more of an autonomous reasoning tool in a way different than what it is, maybe more along the visions of symbolic AI people. But it is incredibly facile at taking some idea and writing a rap song around it or a poem or a shanty song or whatever you might want. Uh, and that is different. I think, uh, what are sometimes called the word cell classes may be under greater threat from AI than I would have predicted. Uh, but again, this hasn't all quite come to fruition yet. We're waiting, and I expect many surprises.

Got it. So, so it's— it sounds like you're not— you're not really sure, and you— and you don't necessarily want to, like, hazard a guess because it's too early to say.

No, I'm very willing to hazard a guess, but I want the, the uncertainty brackets to be understood.

Okay.

So my best guess is this: in the short run, it will be quite egalitarian, as people who can't do things well at all now can do them pretty capably. Say, like writing an online essay to get into college. The smartest kids already could do that well. GPT doesn't help them that much, and they were going to work hard on it anyway. Uh, but I suspect the longer-run effects will be less egalitarian, that as people learn how to use these things and they get better, that more, more people will use AI to sort of help them build out projects. So the people who can start projects will be the major beneficiaries. They'll have better record keepers and translators and mathematicians and coaches and colleagues and advice givers. But large language models, they're not a source of actually doing the idea. You can use them to get ideas, but they don't make the decision to go out and do the idea.

Yeah.

So some kind of hyped-up executive function maybe is what will be rewarded, say, over a 10-year horizon.

Yeah.

And that could be inegalitarian because it's talented people who have a lot of project ideas anyway who might benefit the most. That's my best guess.

I think that makes sense. And so to sort of summarize what I, what I heard from you is Because right now what it can do is it can give people skills to do tasks that they would ordinarily not have been able to do at all. So like maybe you're not really a programmer and it can help you build a very, very simple version of an app. Or maybe you couldn't really write a college essay, but it can do that for you. So it can bring the sort of the bottom quartile of skills up.

Even bottom 60, 70%, right? Most people cannot write a college essay. Most people can't finish college.

Okay, right. So, so it can, it can, it can augment those people's skills to let them do something that they couldn't do before. And then for, for other people who are, who are more skilled in a particular domain, let's say a highly skilled programmer, it makes them slightly better, but not— it's not going to necessarily make them do things that they could never possibly do before. And that's the, that's the immediate effect, is what you're saying?

Yes.

Right. And then the set— and then the sort of secondary effect, um, is something like, uh, it enables people, uh, to who want to make things in the world to like go, to go and make those things with the, with the intelligence that it, that it, uh, that it unlocks. So like one of the things, one of the ways I've been talking about it is, is like the allocation economy. It's like people who know how to allocate resources. Um, so those are people who have like managerial skills or investing skills. Are going to be able to deploy those much more quickly and cheaply by using AI instead of using, like, building a human organization.

That's right. Yeah, I think there's another longer-run effect that is not at all in place now, but at some point will be, and that is when we can use AI to evaluate people's talents. Uh, and that will be highly meritocratic in a narrow way. I'm not sure we'll all be happy with it, To be told just how good you are at something is perhaps unsettling.

Yeah.

And on net diminishes human happiness. But we will find potentially successful people much more easily at some point that is not in place now.

I definitely find that it's really quite good for helping me understand who I am, like giving me a picture of who I am. So like, I'll often do things like I'll take long journal entries and I'll throw them into ChatGPT or into Claude and I'll say like, hey, What are the psychological patterns that you're noticing here? And it's really good at taking them out and putting them into words for me. And I think having that reflection, if you ask for it, is really powerful.

I'm about to turn 62, so that's maybe of lower value to me, but I agree with your point about large language models. They're wonderful therapists and analysts. And they can be remarkably objective when that's what you ask for.

Yeah, yeah. I want to just— this is, this is a sort of, this is a sort of slight tangent, but I want to just dig in on the sort of allocation economy idea that we've been talking about because I've— I'm writing an article about this for Friday, actually. And, um, one of the, one of the, I think, interesting implications of that is right now the skills of, of being a manager, um, are not that widely distributed or talked about because there's only a small group of people who are managers. But I think those skills may need to be more widely distributed in the near future as everybody is moving up a layer of abstraction. So there are even junior employees, for example, are thinking about things like, for example, a really common management problem is like, how far into the details should I go? Um, like, I asked someone to do something, do I micromanage them or do I let them go do it but then it comes back wrong? Like, that's a, that's a specific skill that managers have to learn that junior employees don't have to learn that I think might be valuable in an economy like this. Have you thought more about, like, what are the specific— if you're, if we're thinking about allocation or allocating resources as, like, as a core skill of the next generation, have you thought about the, like, specific ways or skills that are bundled under there that are gonna be important.

That's a very good point. Also, if you look at even the top management consulting firms, the kind of advice they give to many people, it seems rather anodyne and boring and repetitive and clichéd. And that's a sign GPT models can do it really well. Yeah. And I think they can. So that might just end up automated and you might still need the consulting firm to make it stick or to be the focal voice putting the message forward.

Yeah.

But the actual amount of labor you need doing that. So it's like everyone will have free McKinsey-level advice or nearly free. But another thing I worry about, as you know, there's a lot of talk about AI and bio risk. What about developing dangerous pathogens? I think it's a big constraint on terror organizations that they're very poorly run.

Hmm.

That's a much bigger problem than, oh, they can't find the optimal pathogen to do us all in, right? They just don't succeed or even try very often. So if they have access to AI giving them better management advice, I think that's actually one of the biggest risks of AI technology.

That is fascinating.

Not the pathogen, just the— here's how you run a successful terror group. And maybe current models are sufficiently protective that they won't tell you, though even there you can probably break through the dam. But there'll be very good AIs that will tell you that pretty soon, right?

Yeah, I think that that's such an interesting underappreciated risk because, yeah, I think anyone that's thinking about AI risk is really interested in the hard problem of like, here's how you assemble chemicals in a specific way to like make a bomb. Um, but the, the human coordination problem is actually like a significant deterrent for people doing bad things. Um, human coordination, deterrent. Yeah, yeah. And human coordination is like one of the things that I think AI is going to be really good at. Um, and, and we see that people talking about that already in, in the, in terms of business. Like, you even yourself wrote You wrote an article recently about ChatGPT and your career trajectory, and what you said is that small integrated teams will produce the next influential big thing, which I think you meant you can do a lot more with AI with a small team, and I think that's a really positive thing, but the negative is that you can do a lot more harm.

That's right. A lot of companies might become much smaller but still rather potent. Yeah, I think Midjourney, when it first had its breakthrough, had what, 7 or 8 people working there? I read, uh, it may be more now, but it's not going to be so many more.

Yeah, it's very small. Um, I'm kind of curious, like, on, on that, on that point, um, one of the things that you've written a lot about Tyler Cowen, and you, you wrote about this in, in a previous book, um, called, uh, The Age of the Infowar, is, um, there are certain skills that are super valuable in the knowledge economy, um, specifically like ordering knowledge, um, and also like being very attuned to being able to remember and use small bits of information within a specific subject area that you associate with people who are autistic. Um, and, and, and that book in particular talks a lot about, um, why those traits are actually quite valuable, uh, in this current era. And one of the things that strikes me is like, that's also a lot of the things that like ChatGPT or, or other AI tools are, are fairly good at. And I'm curious how that changes or what that does to your perspective to like to look at it that way.

I think in the short to medium term AI accelerates the value of those skills, ordering information, grasping how things fit together, knowing a lot, because you now have this extra way of learning. But there could be some point much further out where the AI simply does all the work and the value of that skill becomes quite low. And skills like charisma and executive function are what rise in importance.

Yeah.

But I think to get to that point, we would need AIs really incorporated into major workflows and systems. In a way that's fairly distant. So I don't think it's, oh, just a few more years and then this is going to flip. I think for the foreseeable future, uh, it will help the infowars, GPT models.

And so I, I assume you, you, you include yourself in that?

Like, sure, of course.

Yeah. Are you thinking for yourself, like, about upping your, your charisma level, or you feel confident enough that it's, it's far enough out that you don't need to really worry about that? Or maybe you feel charismatic enough you don't need it?

Well, that's for the world to judge. I will say I've decided to do more personal appearances, both as a way of projecting but also as a way of learning things that I can't learn from, you know, say, a large language model. So it's definitely influenced my behavior already.

Okay, cool. Uh, I'd love to start to get into some of the practical aspects of this, um, which is, yeah, more specifically how you use ChatGPT And before we— like, you're, you're already in, in a screen share. Um, but before we, we dive into it, I, I'm kind of curious, like, how would you summarize at a high level the way that it fits into your life and your work?

There are two quite distinct ways I use ChatGPT. One is on iPhone and the other's on my laptop, and they're totally different. So I don't know if you want to cover both or take them in sequence. But the iPhone is pretty simple. Do you want to start with that?

Let's do it.

So if I'm in a foreign country, I was in Tokyo, don't speak Japanese. Not many people there speak English. I use it as my universal translator. It will get better than that, but it's fine already. Yeah, it's not Star Trek great, but it's amazing. And the other thing I do is I use it to read menus or even just tell me what should I order.

Yeah.

So I was in Buenos Aires. I was at a Paraguayan restaurant. Paraguay is one of the countries I've never been to. So I'm an idiot when it comes to Paraguayan food. I took a photo of the menu asking, well, GPT, what should I order here? Which are the classic dishes? And it tells me. Now that's amazing. I was in Honduras a few days ago. You see a plant, you see a bird you don't know. If you can take a photo of it, you take the photo, you ask, well, what's this? And it tells you, right? So I use it for that mostly when I travel. Again, that's distinct from my main uses. Yeah, but I'm quite sure it will be enduring.

What do you think that that opens up for you? Like, what is that— what is that like to be able to take a photo of a bird or take a photo of a menu and just like know what the menu says or know what the bird is?

Well, you just learned something, right? But a big part of the value is the interactive experience with the other person you're with that you in some way are discovering this together makes it more memorable. And I think you're more likely to do it with another person or people than if you're alone.

Yeah.

So it adds flavor to the trip itself. Like you have these now multiple modes of how you discover things.

That's really interesting. It reminds me, um, I don't know if you know Andrew Mason. He's the former, um, he's the former CEO of Groupon and he runs Descript, the, the podcasting software company.

I might have met him once, but I don't know him.

Yeah, he used to run this company that was— it did like audio tours of cities. And so what you would do is— and they were all, I think, created by the company and then eventually created by like third-party tour guides— where, wherever you were, like I did this in Rome, you could, um, walk around and it turned the whole city into an audio tour, sort of similar to like the one that you go on in a museum. But it was the entire city and you could pick one that's like history or food or architecture or whatever, and you would walk to a specific spot and then it would say like, look to your left and this is a building and here's everything about the building. And it— that company didn't work, but it was a, it was a pretty magical product experience. And it sort of strikes me that in some ways ChatGPT is that without needing it to be a separate company.

Yes. And if you're sort of on the road out there and there's some fact you want to know, like, oh, what's the population of the city? Uh, it's easier and quicker than Google or Wikipedia. Yeah, maybe it's a small edge, but for, for something as important as that, I'll take a small edge. Yeah, I just ask it, right?

What if you— like, this, this actually reminds me of, um, I— before we recorded, I, I, I asked Twitter for, uh, questions for people that people wanted to ask you. And, um, Patrick McKenzie, who you interviewed on your podcast recently, asked me to ask you, how do you check the output for correctness workflow-wise, and where do you feel comfortable not doing this? And I think that's, that's kind of related to this example because like, particularly in a, in a country like Paraguay or Japan where you don't speak the language, um, it's, you're, it's like hard to actually check the output. So like, do you have any specific rules or heuristics for like when you, when you trust the output and when you don't?

I worry about hallucinations a lot less than most LLM users, I think. So a lot of my questions, oh, what bird is this? Let's say it's wrong and it's just pulling my leg. Okay. Like, I didn't know what bird it was and now I think it's the wrong bird. Like, who cares? I'll just ask it again and move on. But pretty often I'll just ask it, are you sure that's the right answer? Or please correct any hallucination. And most of the time, if there's a problem, it does correct it. Not all of the time. I'd guesstimate 80 to 90%. If you just ask it again, it'll correct an error.

Yeah.

So that's one layer of check. Uh, I think also the areas where I tend to use it— this is now LaptopGPT— it just makes way fewer errors than the areas where a lot of other people tend to use it. So that also limits the degree of hallucination.

How do you characterize the areas where it makes fewer errors?

I couldn't give you a general answer. Yeah, but I can tell you I'm typically using it to learn obscure history.

Hmm.

And obscure history, maybe by the nature of the questions, you're sort of pointing it in the direction of an intelligent part of the information space. It's not too crazy, not too controversial. Yeah. If you ask about the old Byzantine Empire, It does pretty well. I did a whole podcast where I made believe I was talking with Jonathan Swift, you know, the Irish writer, 17th and 18th century. I asked it a series of detailed probing questions for an hour. This has been put online. It didn't make any mistakes. Not one. And there were no redos or no trial runs. I just did it. And what I did on the first try was the final output. So presumably it's read a lot of Swift, read a lot about Swift. It's obscure and detailed enough. Maybe it just ends up boxed into the corners of truth a lot.

Yeah.

So for me, you know, I know how you can get it to hallucinate if you just ask it, what are the 3 best books to read on Jonathan Swift? My best guess is one of the 3 answers won't exist. I understand that, but I don't run into that so often because I'm not asking it that question.

Right, that's really interesting. Um, I definitely have found that too. I haven't used it for obscure history, but I use it a lot actually to read old books. So, um, one example, I was reading Moby Dick recently, and it's really helpful to just take a picture of a passage, ask it to explain what's going on, or even to like visualize the passage where you're just like, hey, can you, can you show me what this looks like with DALL·E? And I think the reason why it's really good for that is it has access to all the text. It's read all the text and there's a lot of supporting— there's many decades of supporting commentary, so it knows what to say.

That's right.

Whereas for more recent stuff, it's more likely to be copyrighted and less likely for it to know, have read it.

Another thing I use it for a lot where hallucinations are not a risk. So I have my own podcast, Conversations with Tyler. I have guests on, I wanna learn the background context to what they do.

Yeah.

So I had one interview with Lazarus Lake. He runs ultramarathons. He hasn't really written anything. There's not that much written on ultramarathons.

Yeah.

That's readily available. But if you keep on asking the thing questions about ultramarathons, you acquire all this background context. Maybe some of it's wrong. It's not really going to matter. I'm the one asking questions, not answering them, and I just feel I know my way around the topic pretty well. Yeah, I'm learning now about the insurance industry. We can talk about that more, which actually the published literature on economics of insurance and its history, it's very bad and sparse for whatever reason. If I keep on asking GPT again, I'm not convinced the answers are perfect,, but I know I'm doing better than I would do any other way.

Right.

And I'm not the one required to know the exact fact. What I want is context.

That makes sense. Yeah. I guess when you're using it, for example, in the insurance area and you say you're not the one that is required to know the exact fact, like, don't you get sort of nervous that you're going to have something in your head that's like kind of slightly wrong?

Well, I already do, right? That's the problem. I could show you one of the things I asked it if you want to look at something concrete.

Yeah, I want to sort of set the stage here really quick. So we're going into your use of LaptopGPT.

Yes.

And one thing that I think is really important to note is you're not in ChatGPT, you're in the playground. And so before we, we head into a specific chat, Why do you use— why do you use the Playground? Tell us.

I don't have any good answer. I mean, I'm stupid, basically. I like the fact that it doesn't look like a consumer product and it works for me. And the maximum length I can slide across, make it longer, shorter. Maybe it doesn't even matter. Yeah, but I just enjoy that and I'm used to the look.

Yeah.

So I use it. Do you think it's a mistake for me to use it?

I actually don't think it's a mistake. I mean, you're, you're sacrificing certain things, right? So like, uh, you don't have access to Code Interpreter, you don't have access to DALL·E, uh, there's no unified experience between mobile and your desktop. Um, so there's a bunch of things that are not as good about the Playground, but one of the things that is going to be better about the Playground is you have a lot more control over the output. So like, you can see right, right on your screen, you can see that you can set the system message. So the system message, for people who don't know, is gonna set the kind of like the personality of how GPT interacts with you. And setting that can be like really helpful. On the right, you can also see, you can set the model, you can set the temperature. So temperature is sort of like creativity for the model, how likely it is to be factual, how likely it is to sort of like make stuff up. You can set the max length, all that kind of stuff. So you have a lot more control. And then another thing that I think is probably true of the Playground that's not true of ChatGPT is the content restrictions are going to be looser in the Playground because it's built for developers and not for consumers. And so they're going to be less careful about what the model says because it's only being used by a certain group of people who are using to make, make products instead of like hundreds of millions of people who are using it every day for random stuff.

So I'm not stupid. That's nice to hear. I mean, I have DALL-E and all the rest. I just don't use them much. Uh, the images are not my main interest. I think it's amazing Uh, but I'm not sitting there playing around with DALL-E. I don't think it's a waste of my time.

Yeah. Um, okay, so, so tell us, so let's, let's go into, into the first chat. You, you have a chat right here. I think you're, I think it's about insurance. Tell us about like, um, tell us about how you did this chat, what, what was the motivation behind it, and then, and how you started it.

Well, Alex Tabarrok and I agreed that we would be recording a podcast about the history of insurance and insurance economics.

Mm-hmm.

And I'm really not well informed about that area, in part because the literature is not that good. I do read a lot. So one question I had was about this difference between whole term and regular life insurance. Uh, the whole product is like a savings investment bundled with the insurance itself, and the other is just the plain old life insurance. So that distinction between the two kinds of insurance, when did that arise? I don't know. Probably I still don't know, but I thought I would ask GPT and you can see my question on the screen. I can scroll down and show you the answer.

So to read the question, you said, when did the distinction between the whole term— between whole term and plain life insurance arise historically?

Right.

It looks like, uh, GPT said the distinction between whole Whole life and term life insurance evolved over time with the development of the life insurance industry. Term life is designed to provide coverage for a specific period with no savings. So we're getting some definitions. Whole life insurance, uh, offering a death benefit along with the savings component, allowing the policy to accumulate a cash value, became more common in the 19th century as the industry matured. Um, so yeah, what did you think of that answer? Like, did it Did it give you what you wanted?

Well, I don't know if it's true. So much of the insurance industry matured in the United States after the Civil War. So if that distinction came at that time, it's at least plausibly the case. And then it says, well, the distinction became clearer in the 20th century as life insurance markets became more diverse. Yeah, I thought it was a good answer. I hope it's a good answer. I'm not going to flat out assert that in a dialogue. But if it had said, oh, that was there from the beginning in the 17th century, or it came only in 1964, yeah, then I would definitely look more into it and try to learn something quite specific. But right now it's just telling me it's following some general patterns I might have expected anyway. That's useful information. I probably won't use it directly. But then I go on to my next question. Can you see that now on the screen?

See it? Yeah. So you said, why would anyone want life insurance bundled with the savings return? Doesn't it mean inferior returns? Tell me, tell me about that. Why'd you ask that question?

Well, it's a common piece of advice for Americans investing that you should not invest through your life insurance policy, that you can do better with a direct purchase of equities. Through a low-cost diversified mutual fund, right? That's pretty standard advice. So I'm asking GPT, why would anyone want to save through their life insurance? Which is a known question. So it gives 8 reasons, a few I hadn't thought of at all. One or two may not be good reasons, but they're probably actual reasons for particular human beings. Yeah. And it's a good answer. What can I say? I learned something. I don't think it screwed up. That was my answer, right?

Yeah. How do you think, um, how would you summarize the kinds of answers? It seems like there's, there's some things you would have thought of, some things you wouldn't have thought of, and some things you think are not that great. Like, what is the, what is the value of that for you?

It spurs my thoughts. I get additional bits of information. Uh, I, one thing I didn't do, but what I would then continue to do, is ask it a question like, well, if someone were writing a criticism of whole term life insurance?

Yeah.

What points against it would they make?

Yeah.

And my guess is that answer will be quite strong.

Mhm. Do you have any—

I don't feel I need to ask it partly, I know, but, uh, that would be the sort of follow-up if I had just done this.

You think we could try it?

Sure, let's try it. We can talk about what's in between also.

Yeah, we can go back up in, in a sec, but I'm sort of curious about that.

If you were to write a critique of why individuals should not buy whole term life insurance, which points would you stress? Please answer like a sophisticated investment advisor. Those add-ons, you don't need as much as you used to, I find, but they're not going to hurt.

Interesting. Tell me about that. Tell me about asking it to sort of simulate a sophisticated investment advisor.

Again, you're pointing it to a more intelligent part of the information space. So if I ask it a question, say from economics, what is inflation? The answer is not wrong, but it's not really better than Wikipedia because the question's too general. If I ask it a question, what is inflation, answer as would Milton Friedman, you get a better answer usually. It's not nothing to do with agreeing with Milton Friedman or not, though. I mean, you might— I guess I do, uh, you're just again pointing it towards smarter bits, you know, in the matrices somehow, right? And what the stuff it knows connected with Milton Friedman is smarter than the stuff not connected to Friedman, right?

So as you get more specific, you're going to get more specific detailed responses, which is what you want.

That's right. And it loves compare and contrast. Even if you don't want to compare and contrast, you often get the best data by asking it to compare and contrast because it's like crossing two swords. And maybe something about the autoregressions, they just get better.

Mm-hmm. Mm-hmm.

Interesting. So here's the critique. There's some general blah, blah, blah at the top. Uh, cost efficiency, whole term life insurance has higher premia. Uh, investment flexibility, it's not as flexible. Less transparency, less control. That's true. You're less liquid, it mentions. That's number 4. Uh, 5, it's comparing the tax considerations to a Roth IRA. Excellent point. I hadn't explicitly thought of that. The whole term can have more financial complexity, higher opportunity cost, which is the key main point. Could be invested elsewhere. I'd like to see it mention equities there, but okay, it didn't. Risk diversification, the insurance company may not be optimally diversified. That's also true. You know, on any exam, I'd give this answer an A, maybe not A+, but a solid A.

Right.

It's all correct. Nothing wrong with it. It did great.

One thing that we might want to try, which could be, could be interesting, is if you delete this specific answer, um, yeah, which I think you can, you can do pretty easily.

Um, you'll have to tell me because like I said before, I'm stupid.

Yeah, yeah, scroll over it, click into the assistant answer. There should be a minus button there.

Learn something new every day.

There you go, minus button. There you go.

Okay, so I got rid of it.

Yeah, so what I want you to do now— so I think it'd be fun to play with the temperature. So right now the temperature is 1, um, but if you move it up, it'll get more creative. Yeah, um, it might be interesting to see if you go all the way, all the way out, what does it do?

Oh yeah. This, this is something they're going to ban on Twitter.

Let's try. It'll become Grok. See, there you go.

It's too creative, right?

Too creative. Um, but I think that's, that's an interesting reflection. You can, you can try it again with a slightly less— yeah, slightly less temperature, like maybe midway. Um, and if you just, uh, click the minus button again.

Yep.

And then ask again, uh, press, uh, press cancel. See where it says cancel?

Yep.

Yeah, and then submit.

Submit. Opportunity cost comes first now, which is, I would say, better.

Hmm, interesting. So you think it's maybe slightly better so far?

Slightly better. It's the same points. Yeah, uh, but in a better prioritized order.

And then it went off the rails.

Nonsense. Now it's off the rails, but that's fine.

Yeah, I think that that's, I think that's a really interesting thing about using the playground is you get to do this and you get to see these are some of the rough edges that like they don't really want you to see in a, in a consumer app. Um, but it does allow you to sort of explore the space of possibilities and the, the upside, the downside of exploring a space. Play the space of possibilities is that you sometimes get junk, but the, the upside is you might, you might find stuff that no one has found before. Um, and in this case, it like, before it went off the rails, it actually gave you a slightly better answer. And I think that's an interesting trade-off for people to consider when they're thinking about when to use ChatGPT or when to use something like the Playground.

If I used it for poetry, I would use the higher temperatures more, right? I've experimented only a bit, but I'm actually pretty fine with it at 1.

Right. Yeah, that seems like it makes sense for you. I'm curious like— Yeah.

Obliterate this. We can talk about some of the other queries in here.

Let's do it. Yeah.

Okay. So here's one where I couldn't quite pull the teeth out. So I was reading a book about the Byzantine Empire, area I don't know much about, and it mentioned inflation under Emperor Aurelian. Which is the early part, is still Roman Empire times before it splits. And I was like, okay, well, how high was that inflation? I wanted to know. I really wanted to know. I didn't just do it for this show. And I asked it and it understood the query very well. And it told me all about the limited data. It may just be the best answer. Maybe we don't know. We just know of texts where people complain inflation was high. It mentions historian Richard Duncan Jones.

Mm-hmm.

I have an imperfect recollection that his is the right name to cite. So that's helpful. I recognize it, but wouldn't have thought of it. So I could put that into scholar.google.

Mm-hmm.

But then at the end it says quantifying it with any exact figure, duh, duh, duh, duh, duh. You know, it's like falling asleep on the job, right? Not willing to commit. So what do I ask? If you had to guess the specific number, maybe I should now be demanding my money back, right? Sometimes that works. Now it just goes into full evasive mode. As an AI developed, right? We don't even need to talk through that. Everyone listening knows the deal, right? Then more blah, blah, blah. Basically, it should have just said, I'm not going to tell you. Uh, but here, finally, some scholarly estimates say it might have been over 1,000% over the course of half a century. Now, I don't believe that. I think that has a decent chance of being a hallucination, but it told me something. And it's like, some estimates, we're not sure. Like, I know we're not sure, so the hallucination may not be worse than the hallucinations of, you know, my fellow economic historians. That's okay, I'll live with that. Yeah, if it had just said, oh, we think it was 30%, Uh, maybe I wouldn't trust that, but it said something.

Okay, this, this actually reminds me of, of, of a really kind of important point going back to, you know, talking about the skills of managers being more important in this, in, in this new world. Um, so like one of the skills of a manager that I think you're, you are picking up on and you already have because you do this, I think, probably with research assistants, is you have a sense for when you get a fact like that, 1000%, it's probably not right. There are some things in that answer that are probably right and some things that are not right. And I think there are a lot of people who are evaluating ChatGPT that see that and they're immediately like, well, I can't trust that. I'm not going to use it. It's not useful. Right? But what managers of people know how to do generally is to take an answer from someone and know which details are like maybe they need to follow up on and maybe the ones that they they don't. And that is a skill that you are deploying here to hold lightly to the things that, that, that don't— that sort of trigger your sense of, I should probably check that. And then maybe, I don't know if you did this in this chat, maybe like go deeper on it, or maybe switch to another tool to like follow up. And that's an example of, I think, a skill that managers have to know that junior employees don't generally have to as much.

That's right. Now I might try Perplexity AI at this point, right? Which is supposed to be great.

Do you use it?

Sure.

Yeah.

Under Aurelian, what was the rate of price inflation? And I'm clicking and I'll tell you the answer. We go here, we go into Perplexity.

There you go.

Some sources say 3% a year, others say 5% a year. Debasement of the currency. Now, one of the great things about Perplexity is that it gives you citations, and I would track those down. And, uh, one is Reddit, eh, the other's JSTOR. I would look at JSTOR. Most of all, I know who it is I should ask, and if I needed to know, I would do that, right? That's how my process would proceed. I would next try perplexity rather than torturing GPT anymore, because I know sometimes when you torture it, uh, it's like, no, the dog just doesn't want to go for a walk, right?

Right.

So that's interesting playground.

So like, let me, let me just— let's unpack that. Let's stick with that for like a sec. So it sounds like the stack for information gathering is something like ChatGPT, then Perplexity, then maybe the sources from Perplexity, then people. Does that sound about right?

Often. Now, if I need to know something quite particular to, say, finish a piece of work, I might start with the person, of course. And there's an advantage to starting with the person. The person may be delayed in responding, but you know the other things won't be. So that which will delay, you ask first to cut down on the length of the delay.

Right. There's something—

logically makes sense, right?

There's something really important missing from that list, which is Google. When do you use Google?

I use Google when I need links, which is quite often. So I still use Google a lot, but I use Google less and less for information. But when I write a column for Bloomberg or do a blog post, I need a link.

Mm-hmm.

For getting the clickable link, Google is still clearly best by far.

Got it.

But not for learning things.

Not for learning things.

Now the link may lead you to what you wanna learn. I get that. Uh, but Google is for links.

Google is for links. AI is for learning.

AI's for learning. Perplexity is for references and sometimes has context that GPT doesn't because it's looking to tie it to references. Mm-hmm.

And why do you think you start with ChatGPT versus starting with Perplexity?

It depends on the query. If it's something recent and I want to know the citation above all else, I might start with Perplexity. But I've only been using Perplexity a few weeks, uh, and just my patterns with it will still evolve, right? So it's a little less familiar.

It's pretty new. It's really having a moment right now. Like, it's— they raised a, they raised a big round, and, and a lot of the guests on the show have been using it. I've started using it more My dad's using it, which is actually like usually a big sign for me. He was using Zoom like years before anyone else. And so he just, he's sort of like the canary in the coal mine for me of new, new interesting software. So going mainstream. So yeah, I'm really interested in following their trajectory.

Yeah. Someone should try to buy them if they can.

Yeah.

Um, now here's a question where I got hallucinations. Do you want to do one of these?

Please, let's do that.

So on Twitter, Jordan Schneider asked me, and here you can see on the screen, what is the best book on the civil service reforms of the Progressive Era as related to the Pendleton Act? Some detail up further on the screen, but don't worry about that.

Yeah.

And I, I asked GPT, but I knew that wasn't the way to go. I just thought, well, I'll do that as an experiment.

Mhm.

In part for this program.

Mhm.

And the first book it mentions, Pendleton Act of 1883, by Jonathan Grotzinger. It's not on Amazon. Uh, I'm not sure it exists. Probably it doesn't exist.

Uh-huh.

It's sort of the book I would want, but it doesn't exist. Uh, this other book, Reforming the Civil Service, I couldn't find. So it's hallucinating more than usual. Then it somewhere, maybe it was the query one up, it recommended Robert Wiebe. Yeah, The Search for Order, which is quite a good book. And it's a good book for me to recommend, not mainly on civil service, but that was something where it jogged my memory. So mostly it failed on that question. But again, I would know to begin with not to ask it that. It's asking for trouble.

I've personally found it to be stunningly good for book recommendations. And in particular, what I like to do with it is have it help me to identify my taste. So I'll throw in a bunch of books and then be like, "What are the commonalities between these books?" And it'll be like, "Here's the kind of stuff you like." And then I'll push it further and be like, "Okay, tell me more stuff that I haven't heard of." And I've found a number of books that I've that I absolutely love from it.

I use people for that basically, and that works very well for me.

That's also good. Yeah. Yeah. If you like, if you like a person liking their book recommendations, you're pretty, you're pretty likely to like their book recommendations. Well, let's move on. Do you have other chats you want to show me?

Well, let me see what I have in here. So I need to bring you back to screen. The Playground. Oh, we did Aurelian. Yes, it is. Again, I was reading the book on the Byzantine Empire, and Chrysostom is mentioned, and I know I've known about him, but it's rusty knowledge to me, so I just thought I would ask about him and get more context. Right. Who was he and why was he important?

So you're kind of— yeah, go for it.

The answer seemed quite good to me. Again, it's obscure history.

Yeah.

And it's working.

So you're kind of using it as almost like a second screen for reading where—

exactly—

you're reading a primary source and you want more details or insight into it, and, and you're just, you're turning to ChatGPT for that.

That's right.

Yeah.

Now another thing we'll use it for a bit. This is typically with my wife. We'll use it for the dog. So we have a dog. It was given to us by our daughter who now has kids. So we haven't had the dog for long. We're not dog experts. So there's different questions about the dog. Like, will the dog get fleas? Or if the dog is limping, should we worry? Or how much, you know, should the dog be eating? Or Well, everything about the dog.

Yeah.

And that's useful. I, I think it, it must know a lot about dogs.

Yeah, I'm sure it does. Um, I, I do that too. I do that. Um, not, not necessarily— I don't have a dog, but for like basic health questions and stuff like that, sometimes it won't answer, but a lot of times it gives me like a general sense, which is nice. Um, so, so far what we've done is we've covered— I think we've covered a decent amount about how you read with it. So reading with it as a second screen. I think we've covered a little bit about how you write with it. So doing a little bit of extra research on topics that you might not be as familiar with or sort of getting the lay of the land on topics. I'm curious if there's anything else writing-wise that you've started to use it for or if that's really the extent of it.

That's most of it. I think just as a pure writer, Anthropic is better, but I don't use it to write anything I do. I'm really very, very careful not to do that.

Mm-hmm.

Uh, I just think it's a bad habit and it's in many cases unethical or even illegal. Uh, but I don't do it, don't want to do it. But if I were doing it for whatever reason, I would use Anthropic.

Got it.

It sounds more natural and more human.

That makes sense. Yeah, I, I find the same thing. I think, I think, um, Claude is a bit better at capturing a specific tone, um, that you're going for if you, if you need to use it for that. Um, and then I'm curious if there's any way in which you use it or don't use it in the classroom.

Well, I'm teaching a class right now. We started last night, and, uh, GPT-4 is on the reading list. Everyone in the class is required to pay the $20 a month to subscribe to it. The class is History of Economic Thought, so it's mostly a class in obscure history. And I think it will work very well for the students, but the class just started. Now, I taught a class the year before to law students where I made them all write one of their three papers using GPT.

Mm-hmm.

Not solely, but you and GPT together figure out how to write the paper. And that was very successful. The papers generally were good. People liked the experience. They learned like limits and scope of GPT.

Yeah.

And, uh, that's how I've used it in the classroom also.

That's interesting. So it seems like you're, you're leaning into it versus banning it.

Um, oh, of course. And I told the students last night, I was like, look, if you want to quote unquote plagiarize with this thing, your paper still won't be great. So like, don't do it. 'And probably I'll know you did it. I know enough about this stuff, but just figure out some way to use it wisely and in a smart manner. And you're going to learn things about it that I don't know. And yeah, just figure it out,' is what I told them. I said, 'I'm not going to bust you for being on one side of some vague line or another. Just don't do something stupid you could ever get in trouble for, and it should be fine.' And I think that's going to work out. I'm not worried.

That's really interesting. Have you found any, any specific or interesting ways that students are using it that you wouldn't have predicted that are helpful for them?

Well, foreign students, especially from China, use it to improve their English, and it has a big, big effect.

Yeah.

Not just a marginal effect. And I don't think they even always know that maybe they're not supposed to, or maybe it's allowed, or maybe it's a gray area. Personally, I think it's fine. Universities deal with this in different ways, but I've seen people go from like bottom 10% of writing skill to top 10% overnight. There's a risk, right? That becomes a crutch and you never learn a lot about English.

Yeah, I've seen the same thing. I think there are a lot— a lot of business obviously is conducted in written form, and there are people that I work with or work for me who speak great English, but writing it is a little bit challenging and you can sort of tell. But the minute ChatGPT came out, their writing turned perfect and it was so helpful, you know? And it made me think about, 'cause I think that that's actually true in a lot of other areas where a lot of people, the people I'm talking about, like they could write in English, but it just, you could tell it wasn't totally right. But I think that, and they just needed this like little subtle translation from their level of English to like natural fluent English. But I think that there's actually a lot of other of those kinds of translations that have to go on that GPT facilitates. So like an example is my dad, who I brought up earlier. He's not, he's not a particularly like technical guy. But he has an idea for an app. And usually when he does that, when he has an idea, he'll go to me and say, hey, I need you to write this up for me so I can hire someone to like do it. Um, but with GPT, he can actually just like say his idea and it will write a product spec that looks like it's written by a product manager. And that kind of like translation, it's all in English, it's an English to English translation, but it's, it's a translation from you know, his level of technical fluency to a product manager level technical fluency is like super helpful and important because previously he wouldn't have necessarily been able to do that. And I think people sort of miss that when they think about GPT's translation capabilities.

Context is that which is scarce, as I like to say.

Yeah. So, okay, so we've gone through— I think the classroom stuff is really fascinating. Before this episode, We, we— I tweeted about the fact that you're coming on the show and a bunch of people ask questions. So I want to do a little bit of a rapid-fire question asking of Twitter questions, if that's okay.

Sure.

Cool. So Henrik asks, in the least possible number of dimensions, if ChatGPT stopped existing today and other LLMs, how would that affect your productivity? Would it be less posts on Marginal Revolution? Worse interview questions, something else?

Worse interview questions. I would just know less. I don't even know how to measure my productivity, right? Maybe it wouldn't go down. I would feel much less smart. However, maybe the stupider version of me is more productive.

But much less smart, I think, is a really important, interesting thing to say.

But the returns to being smart in labor markets, they're not as high as you might think. That's why there's a lot of uncertainty in my tone here. So it may just be private consumption being smart. I don't think, say, it would get me more people listening to the podcast. It may get me fewer.

Right. Interesting. What do you think of as the most underrated or overrated use cases for ChatGPT? And James asked that question.

I don't have a good sense of that. I'm not even sure, like, what is rated how. I think the problems of hallucination are somewhat overrated. Uh, that I, I do have a sense of. Things like programming, I don't do. Uh, it just seems to me almost everyone underuses it for most things.

Yeah, that makes sense. Um, uh, Daniel asks, which I think is relevant, how do you manage to switch between reading books and using ChatGPT to interact with the content while still maintaining your reading flow?

Well, it slows you down quite a bit. Yeah, but you're still reading with GPT. You're just reading on a different screen. Uh, but you should do it in areas where you don't know a lot and really do need to learn something, not just do it all the time.

Yeah, yeah, that makes sense. Um, @thisisgray asks, would you use AIs to speak for you online?

I'm not sure I understand the question.

Let's say, would you use an AI? At some point, there will be text-to-speech and vision models that are good enough that you can have a little clone of yourself that might go on a podcast, I think is the thrust of the question. Would you allow a clone of yourself to go on a podcast?

I'm not sure the choice will be up to me. I'd like to see what it's like. I will say this, I published a book online this last year called GOAT: Who Is the Greatest Economist of All Time? And the audiobook version will be read by an AI and it sounds remarkably like me. And I'm happy about that. Not just fine with it, happy about it. The other stuff, I'm not sure.

Yeah. Why are you happy about it?

Reading your entire book out loud is a very costly and taxing exercise. And this means I don't have to do it.

That makes sense. That makes sense. Okay, so we are rounding toward the final part of the interview, which is— it's apropos of this, of the previous question, which is about cloning yourself, because I actually created a little bit of a clone of you. And in this, I want to do a segment with you called the Tyler Test. Which sort of play on the Turing test, where I ask you a question and I'll have you answer it. And then I will also ask the clone that I made the same question. We'll see if it answers the question in the way that you would, and we'll see how good it is. And just for a little background, the way that I made this is you can make a custom version of ChatGPT. You can give it a personality. I think one of the things that makes it likely to work for you is you've written so consistently for so many years that GPT has probably ingested a lot of your writing and we'll see how it does on some of the questions that I have for you. And one important thing is there's no web browsing and I didn't upload any books or anything like that. So it's only what's actually just in GPT, what it knows about you itself.

One thing I would note, my online book, Jeff Olms and I created a RAG-based GPT bot to answer questions about the book. Yeah, it's not claiming to be me. Overall, I think it's pretty good. It's not perfect, it'll get better. We want to upgrade it, but I'm happy with it. I'm glad I did it.

That's cool. I would love to link to that, um, so definitely send that to me afterwards.

Um, that's econgoat.ai.

Perfect. So we are going to get started with this segment. It's called Tyler Bot. Welcome. My first question for you: what are the core lessons of economics?

The first would be incentives matter. The second would be there's always an opportunity cost. Those would be the two most important principles of economic science, and trying to apply them consistently would be the first mark of a good economist. Hmm.

So we, we're asking Tyler Bot And we said, what are the core lessons of economics? Incentives matter. There's no such thing as a free lunch, which I, I have heard you say before.

Yeah, that's the same as opportunity cost, you know. I would give it that answer as correct.

Okay, okay, interesting. So it's going beyond two.

Yeah, the answer is very good.

Very good. How would you rate it out of 10?

I'm not sure what the scale is, uh, but I'd give it an A.

Okay. Um, so that's, so that's number one. That's our first question. Um, second question, and I got this from, uh, from your most recent book.

Too much now. Like, I get that it feels it ought to give more.

Mm-hmm.

But there's something about economics where often less is more. I'm not going to take back the A, but a good answer to that question is actually pretty short.

Let's see if you had to make it shorter, how would you do it? Make it short.

Well, I would just pick the first few.

Yeah. Well, let's see if it knows to do that, because that's— yeah, that's what you would do.

Okay, so the first two are now the same.

Now we're, uh, and the third one is correct, so it's doing this well. And now it's just, now it's just summarizing it in bullet points instead of, uh, so it's going off the rails. It's not quite, it's not quite you, but it's, it did stick.

Yeah, I'd give it good marks here.

Okay, okay, here's, here's my next question, which I took from your book, um, Talent. What are 10 words your spouse would use to describe you?

Curious, works hard, loves to travel. I'm not sure I should mention them all.

Let's see, maybe we can do 5. So you're, you're, you're, you're got 4.

Good father. Nice. And should walk the dog more. It's not going to get that last one.

Okay. What are 10 words your spouse— oh, I say, what are 5 words your spouse would use to describe you? Intellectual. Curious.

Curious.

We got curious. Analytical, eccentric.

Those are good answers. They're not worse than my answers. You know, you can't expect it to know about the dog, right?

The dog isn't there.

That's why we have to ask ChatGPT about the dog. A year from now, we'll know everything about the dog.

Well, you know, I like that it got curious. That's the main overlap, but it missed the travel and good father and it missed the dog. But maybe GPT-5, maybe GPT-5, um, maybe Tyler Cowen, right? What did you say?

It's error or my error. Don't assume the error is in the GPT.

That's true, we can't assume that. Okay, last question. Um, in your view, what is the most underappreciated way in which AI, particularly tools like ChatGPT, will fundamentally alter our understanding of economics and human behavior in the next decade?

I don't think in the next decade it will fundamentally alter our understanding of economics. I think at some more distant point it will be possible to simulate small economies using GPT-like methods, and we might learn a lot from those simulations. Uh, those seem distant with data collection as the constraint, not, not the constraint on the AI side per se. So I don't think it will matter for quite a while.

Okay. Well, let's see. Let's see if GPT Tyler, GP Tyler shares your view. Huh. Do you agree with this? It's saying, I think it will change radically transform our conception of information costs and decision-making processes.

Those might be true claims, but I took you to be asking how will it change economic science and research, and it's answering how will it change economies.

Yeah, I think I more meant it as how it will change the economy, but I think the answer of how it will change economics is a good one.

It doesn't sound like me, this answer. I don't think they're bad answers, but it doesn't seem like a Tyler bot. It seems like it couldn't find what Tyler might think, and it came up with a pretty good generic answer.

Okay, all right. So, so that's— what, what letter grade would you give that one?

Well, for accuracy, it's doing fine. For Tyler-ness, I'd give it a C-.

Okay, C-.

So we got an A, and the middle one was Yeah, we had two excellent ones and then one that's a perfectly fine answer but has no Tyler in it.

There you go. Well, that's, that's the story. Um, uh, before I let you go, anything that you wanted to cover or talk about that, uh, that we didn't get to?

Well, I would just, uh, you know, recommend to people that they read my book EconGoat.ai. They can just put it on their Kindle, print it out, read it on screen, but it's also embedded in an AI. In the form of a chatbot. And I made a deliberate decision to publish a whole book, quote unquote, in GPT-4. And we're working on Claude and also Gemini versions of it. Wow. And I think this is the future for at least some book publishing. Like, why not interact with the thing? It's an option. You don't have to do it. And I'm very excited to have been able to do that.

What have you learned or what have you found so far having done that that you didn't know before?

Well, many people email me or on Twitter, they say what their questions are for the thing and you learn what your readers really care about and it's not what you want them to care about. And that's part of the experiment. So maybe books of the future will in fact use this data and give readers what they really want to care about. I saw a question on Twitter this morning, Someone said they asked the chatbot, what would Tyler think that Milton Friedman would say about Ayn Rand? Which is not really a question about economics. Now, the answer was good. It was both a good answer and a good Tyler answer. But you just learned people want something really quite subjective out of even economics books. And I think that will shape how we write and think over the next few decades.

That's fascinating. I love it. Um, well, I think that's a, that's a great place to leave it. Thank you so much for doing this. It was super fun to have you.

My pleasure. Thank you, Dan.