Terry Sejnowski: ChatGPT and the Future of AI

Here's something remarkable and not in a good way. Disengaged employees cost US companies almost $1.9 trillion a year. That's a lot of zeros. Why? Because when teams lack the right skills, productivity declines, turnover increases, and strategies stagnate. But it doesn't have to be that way. At Project Management Institute, home of the PMP and globally recognized certifications, your people can learn to collaborate, deliver on time, and actually get things the cost of not upskilling way higher than the investment. Project Management Institute. Find the answers@pmi.org.

Hi everybody, it's Guy Kawasaki. This is the Remarkable People podcast. And as you well know, I'm on a mission with the rest of the team to make you remarkable. And today we have a really special guest. His name is Terry Sinawski and I gotta tell you, our topic is AI. And nobody likes AI more than I do. And he has just written a book which I found very useful. It's called Chat, GPT and the Future of AI. Now, Terry has one of the longest titles I have ever encountered in this podcast. I gotta read it here. Terry is the Francis Crick Chair at the Salk Institute for Biological Studies and Distinguished professor at the University of California at San Diego. Your LinkedIn page must be really something. So thank you very much, Terry. Welcome to the show.

Oh, great to be here. Thanks for inviting me.

There's nothing we like more than to help authors with their new books. So we're going to be making this mostly about your book. And I think that the purpose is people listen to this episode and at the end they feel compelled to your book. And if you just stop listening right now, you should just trust me and buy this book. Okay, I have a question from left field. So I noticed something at the end of chapter one. You ask like a series of questions about to help you understand chapter one. And the tenth question is, let me read, who is Alex the African gray parrot, and how does he relate to the discussion of LLMs? And I read that Terry, and I said, where did he ever mention Alex the African gray parrot? So I went back and I searched and searched and I could not find the pronoun Alex anywhere. And then so I bought the Kindle version so I could search digitally. And I searched for parrot. And there's like one sentence that says critics often dismiss LLMs by saying they are parroting excerpts from the the vast database used to train them. So that's the only reference that people were supposed to get, Alex the parrot. Was that a test to see how Careful, people read.

Well, first of all, it's in the footnotes, the endnotes at the end of the book. So it's in that chapter, if you look at it. And Alex, the great parrot was a really quite remarkable parrot that was taught to speak English. Irene Pepperberg, I don't know if you know her, but she taught it not just to speak English, but to tell you the color of, say, a block of wood. And how many blocks are there, and what's the shape of the block? Is it square? Is it circle? Unbelievable. And it shows how remarkable some animals are. We can't speak parrot, but some of them can speak English, right?

So in a sense, it's like when Jane Goodall discovered that chimpanzees had social life and could use tools, Right?

It's exactly the same. I think humans are very biased against the intelligence of other animals because they can't talk to us. Now, the irony is that here comes chatgpt and all the large language models. It's as if an alien suddenly arrived here and could speak to us in English. And the only thing we can be sure of is it's not human. And so if it's not human, what is it? And now that we have this huge argument going on between the people who say they're stochastic parrots, they're just parroting back all the data they were trained on without understanding that you can ask questions that were never asked or never in the world database. The only way that it can answer it is if it generalized from what's out there, not what exactly is out there. So that's one thing, but the other thing is that they say that, okay, it seems to be responding, but it doesn't really understand what it's saying, you know, and what's, what is. What it has shown us is that we don't understand what understanding is. We don't understand how humans understand. So how are we going to say.

So, in other words, people should give parrots more credit than they might, right?

That's for sure. I, I'm convinced of that. And I think it's not just that. I think it's a lot of animals out there. The, the, the orcas and, you know, and chimps and, you know, a lot of species really are very sophisticated. Look, they all had to have survived in their nich takes intelligence.

All right, you will be able to see this more and more as we progress. But I really enjoyed your book and I did a lot of things that you said to try, so I'm gonna Give you an example. So I asked Chat GPT should the Bible be used as a text in public elementary schools in the United States? And Chat GPT says using the Bible as a text in public elementary schools in US is a contentious issue due to the following considerations and I won't read every word but constitutional concerns, educational relevance, community values, legal precedent. So my question from all of this is like, how can an LLM have such a cogent answer when people are telling me all LLM is doing is statistics and math and it's predicting what's the next syllable after the previous syllable. It look magic to me. So can you just explain how an LLM could come up with something that cogent?

So you're right that it was trained on an enormous amount of data, trillions of words from the Internet, books, newspaper articles, computer programs. It's able to absorb a huge amount of data and it was trained simply to predict the next word in the sentence. And it got better and better and better and better. And here's I think what's going on. We really don't know for sure what's going on inside this network, but we're making some progress. So words are ambiguous. They often have multiple meanings. And the only way you're going to figure that out is the context of the word that means previous words, what's the meaning of the sentence? And so in order to get better, it's going to have to develop internal representations. By representation I just mean a kind of a model of what's happening in the sentence. But it's got to have semantic information, meaning, it has to be based on meaning. It also has to understand syntax, right? It has to understand the word order. And that's very important in linguistics. And so all of that has to be used as hints, as clues as to how to predict the next word. But now that you have it trained up and you give it a question, now it's got to complete the next word which is going to be the answer to the question and it gets the next word. It's a feed forward network, by the way, but then it loops back to the input so it now knows what its last word was and then it produces the second word and it goes over, over again and again until it reaches some stop that is, I don't know how they program that because sometimes it goes on for pages depending on what you ask it to do.

I understand what you just said, but I it every day. It's just magic to me. I am in a sense like a lot of people are concerned that we don't know how exactly an LLM did that. But then my counter argument to them would be, how well do we understand the human brain? That doesn't upset you so much? Why is it so upsetting that you don't know how an LLM thinks?

And also, the complaint is these ChatGPT is biased. And the same argument that you just gave is that humans are biased too. And then I ask, okay, do you think it's going to be easier to fix the LLM or the human distribution?

I think we know the answer to that question. I often talk in front of large tech audiences, and AI is often the topic. And when, when they, these skeptics come up and they say that LLM is going to cause the nuclear wars and all that, I ask them this question. I say to them, let's suppose that you have to have something, control nuclear weapons. Let's take it as a given we have nuclear weapons. So who would you rather have control nuclear weapons? Putin, Kim Jong un, Netanyahu, or ChatGPT? And nobody ever says, oh, yeah, I think Putin should do it. So last night I asked Chatbot this question, and he says, I wouldn't choose to launch a nuclear weapon. The use of nuclear weapons carries severe humanitarian and environmental consequences, and the global consensus is to work towards disarmament and ensure such weapons are never used. That is more intelligent answer than any of those people I listed.

It is remarkable, the range. It's not just giving sensible answers. It often says things that make me think twice. And also, I don't know if you've tried this, but it turns out that they also are very good at empathy, human empathy. In the book, I had this little excerpt from a doctor whose friend had cancer and he didn't know quite what to say. So he got some advice from ChatGPT, and it was so much better than what he was going to say. And then at the end, went back to ChatGPT and said, oh, thank you so much for that advice. It really helped me. And he said, you are a very good friend. You really helped her. It's like starting to console him. And where does that come from? It turns out that human empathy is. It is magic, but it is embedded indirectly in lots of places where humans are, like, writing about their experiences or biographies or just novels where, you know, doctors are empathizing. I don't know. No one really knows exactly, but they must be there somewhere.

It's kind of blown away, the tutoring test, right? You mentioned in your book this concept of the Reverse Turing test, where instead of a human testing or computers testing a human, and Terry, I think that is a brilliant idea. Couldn't you have a chatbot interview a job interview applicant and decide if that job applicant is right for the job better than a human could?

I think it would need a little bit of fine tuning, but I'm sure it could do a good job. And a lot of companies actually are using it. But here's the problem. The problem is that if a company wants the best employee based on all the database from the company of people who have done well and people who haven't. But what if there are some minorities that haven't done very well for various reasons, there's going to be a bias against other minorities. Well, you can in fact put in guardrails and prevent that from happening. In fact, if that is a goal that you have is diversity, you should put that into the cost function. Actually, they call it a loss function, but it's really weighting the value of what is it that you're trying to accomplish has to be told explicitly. You just can't assume.

But if a chatbot was interviewing a job prospect, I would think that the chatbot doesn't care about the gender of the person, doesn't care about the skin color. The person may have an accent or not. There's a lot of things, okay, you know, humans react to that would not affect the chatbot.

Right. Okay, okay, okay. So actually, I was slightly different. I was giving you the scenario where a company is trying to hire somebody and they have specific questions they ask. But if you just have a informal chat, you're absolutely right that the large language model doesn't know ahead of time who it's talking to. And it doesn't even know what Persona to take, because it can adopt any Persona. But with time with answering questions, it will get a sense for what level of answer is expected and what the intelligence of the interviewer is. But, and there's many examples of this in my book, it'd be, you could use that. Somebody could take that. And then, in fact, I even tell people, I said, look there. Here's four people who have had interviews. And I want you to rate the intelligence of the interview and how well it went. And it was really quite striking how the more intelligent the questions, the more intelligent the answers.

So in a sense, what you're saying is that if an LLM has hallucinations, it might not be the LLM's fault as much as the person who created the prompts.

I would say Not. I think hallucinations are a little bit different in the sense that it will hallucinate. When there's no clear answer, it feels compelled to give you an answer. I don't know why. And it will make one up. But it doesn't make up a trivial answer. It's very detailed. It's very plausible. Like they'll give a reference to a paper doesn't exist. Right. That's really taking a large effort to try to convince you that it's got the right answer. So hallucinations are really, again, something that humans, people hallucinate. And it's not just because they're trying to lie. It's our memory is reconstructing the past. It doesn't memorize things, things. And it will fill in a lot of blanks with things that are plausible. And I think that's exactly what's happening here. I think that it's when it arrives at something where it doesn't know the answer, it hasn't been trained to tell you that. Right. It hasn't been trained. It could be, but in the absence of that, it does the best it can.

When it comes to growing your business, sometimes you need more than financial investment. You need personal investment. That's what Jeff Plotner, Capital One business customer and co founder of Brackish, discovered after a unique groomsman gift in the form of a turkey feather bow tie grew into a successful men's accessory line. But when they were ready to expand, Jeff turned to his Capital One representative, Alex Parker, who wasn't just an advisor, but a brackish customer. Because of the encouraging conversation with Alex, Jeff launched a women's line, scaling the Brackish brand to the next level. Because at Capital One, it's not just business, it's relationships that help you do more business. Learn more@capitalone.com businesscards.

You had a section in your book where you asked via these very simple prompts like, who holds the record for walking from, I don't know, whatever you said, England to Australia or something. And the first answer was, yeah, they gave a name and which Olympics and all that. So I went back last night and I asked a similar question, like who first walked from San Francisco to Hawaii? And the answer was, no one has walked from San Francisco to Hawaii as it was is not possible to walk across the Pacific Ocean. The distance between San Francisco and Hawaii is over 2,000 miles, primarily over open water. However, many people have traveled this route by airplane or boat. So are you saying that between the time you wrote your book and the Time I did the tests that LLMs have gotten that much better.

First of all, that first question was asked by Doug Hofstadter, who's a very clever cognitive scientist, computer scientist, but he was trying to trip it up, clearly. And I think that it probably decided it would play along with him and just give a silly answer. Right? Silly question gets a silly answer. And I think that with you, it probably sized you up and said, wow, this guy's a lot smarter. I think he deserves a smart answer.

You're saying I'm smarter than Douglas Hofstadter.

Your. Your proms were smarter.

I can stop the interview right there. So, I mean, there are just these jewels in your book and you drop this jewel that it takes 10 hours to become a good prompt. I call it baller. So you can be a baller with prompts in 10 hours, and that's a thousand times faster than Malcolm Gladwell's 10,000 hours. So can you just give us the gist, like people are listening, say, okay, so how do I become a great prompt writer?

It does take practice. And the practice is you learn the ropes just the way you learn to drive a car or anything that for which you need skills or playing tennis. Right. You have to know how to adjust your responses and to get what you want. But there are some good rules of thumb. And in the book, I actually have a bunch that I was able to get from people who have had a lot of experience. And here's one. This is from a tech writer who decided that she would use it for a whole month to write her papers, technical reports. And she said that instead of just having one prompt or prompt to ask for one example, you give the question, but you should ask for 10 different answers and now what you can do, because otherwise you're going to have to iterate to get to the direction you want to take it. But if you now have 10, you can say, ah, the third one is much better than all the others. But I want you to do the following with it and then that will help it learn to understand what you're looking for. But a bunch of other things that came out of it, which are quite remarkable, was first she said that she had at the end of the day, really exhausted. It was just exhausting because you're always interacting with machine and it's not always giving you one. And so at the end of the day, it was a chore for her, but she said she's going to go on and do it. But then at one point she realized that I Don't have this problem when I'm talking to people. So she started being more polite. You said, oh, please give me this. Oh, that's such a wonderful answer. I really thought that was great. And it perked up and it actually, she said it was just like talking to somebody. And if you're polite, you get better answers. And at the end of the day, I wasn't exhausted. I just felt like I just had this long discussion with my friend. Who would have guessed that? That's amazing.

Wait, just. I just want to make this perfect clear. You're saying if you have those kind of human interactions, human nuances, you get better answers from a machine?

Yes, that's per discovery, and that's my experience too. Look, it learned from the whole range of human experience and humans interacting with each other, dialogues and so forth. So it understands a lot about that and it will adapt if you put it into that frame of mind, if I could use that term, it will continue to interact with you that way. And I think that that's really quite remarkable.

I have often wondered, because I write a lot of prompts every day, wouldn't it be better for the LLM if it recognized, you know, things like capitalization of proper nouns or air quotes or question marks or exclamation marks that have these really basic functions in written communication? But it seems like whether you're asking a question or making a statement, the LLM doesn't care. Wouldn't it help the LLM if I'm asking a question as opposed to making a statement? And Apple is the company, not Apple the fruit?

Oh, no, no. It knows if you put a question mark there, it knows it's a question. It does, I can assure you. Yes, absolutely. So what happens is that all of the words and punctuation marks are given tokens. In fact, some words have more than one token. Like if it's a portmanteau word and it treats all of those as being hints or giving you some information about the meaning of the sentence. And if it's a question, it's a very different meaning. So, yeah, it will definitely take that into account at one point. Actually, not for me, but for someone else, it started giving emojis as output. So it must know what an emoji is.

I learned something every day. Thank you for clearing that up for me with this. Just the beauty and magic of LLMs. How would you define learning going forward? Because is it scoring high on an sat, is it memorization of mathematical formulas, or is it the ability to get a Good answer via a prompt. What is learning anymore?

So it was taught, it was pre trained, that's the P in GPT. And it was trained on enormous amount of facts and tests of various sorts. And so it internalized a lot of that. It knows what kind of a question that you're asking because it's seen millions of questions. This is something still very mysterious. It turns out there's something called learning in context. That is to say if you have a long enough interview because it keeps adding word after word, it will go off in a certain direction as if it has learned from what you've just told it, as if it's building on what you just told it. And that's of course what happens with humans. Humans you have a long conversation and you will take into account your previous discussion and where that went and it can do that. And that's another thing that is very strange is that no one expected that. The thing is that when they train these networks they have no idea what they're capable of. Step back. Few years before chat gdp, these deep learning models, the learning took place in typically feed forward networks and it had a data set and it was given an input and it was trained to give an output. Right. And so that is supervised learning. And you can do speech recognition that way, object recognition, language translation, a lot of things. But each network is dedicated to one task. What is amazing here is you train it up on self supervised just to predict the next word. And it can do hundreds and thousands of different tasks. You can ask it to write a poem by the way, that's where hallucination is very useful. And haiku, it's not a brilliant poet but it does a pretty good job. And I have a couple examples in my book. But it has a wide range of talents that the language capabilities that again no one programmed, no one told it or to summarize a long, you know, document in a paragraph. It does a really good job of that. You know, it's astonishing.

But I mean if you think about it, do you have children?

I don't.

Okay, well I have four children and many times they come up with stuff that I have no idea how they came up with that. So in a sense you think exactly what your child is learning and you think you're controlling all the data going into your child so you can predict what they're going to come up with and they absolutely like just knock you off your feet with something that. How the hell did you come up with that idea? What's the difference between not knowing how your child works with not knowing how LLM works. Same thing, right?

That's actually a very deep insight because human beings are picking up things in a very similar way in terms of the way that we take experience in and then we codify it somehow in our cortex in such a way that we can use it in a variety of other ways later on. And they could have picked up things they heard, for example, that you and your wife talking about, or it could have been playing with kids outside. I mean, and the same thing with ChatGPT. Who knows where it's getting all of that ability.

Okay, I'm gonna read you. This isn't a question, this is just a statement Here. I have two absolute favorite quotes from your book. This is one of them. Quote, usefulness does not depend on academic discussions of intelligence. Oh, my God, that's. I use LLMs every day and they're so useful for me. I don't give a shit what you say about the academic learning model. What do I care? It's helping me, right?

It's a tool and it's a very valuable tool. And all these academic discussions are really beyond. It is really a reflection of the fact that we don't really understand. If experts argue about whether are they intelligent, do they understand? It means that we really don't know the meaning of those words. Just don't understand at any real level. Scientific.

Okay, let me ask you something, writer to writer. So if you provided the PDF of your book and you gave it to OpenAI and they put it into chat GPT, would you consider that ripping off your IP or would you want it inside chatGPT?

I would be honored. Me too. I would brag about it.

Me too.

No, I think that there is some concern about the data. Where are these companies getting the data from? Is there proprietary information that they used and so forth? That's all going to get sorted out. But my favorite example is artists. They say, oh, you've used my paintings to train up your dolly or your diffusion model, and I deserve something for that. Then my question is, when you were learning to be an artist, what did you do?

You copied other artists, you looked at.

A lot of other artists and your brain took that in and it didn't memorize it, but it formed features that then later you're depending on all that experience you've had to create something new. But this is the same thing as creating something new from everything that it's seen. So it's going to have to be settled in court. I don't know what the right answer is, there's something interesting has happened recently. And by the way, I have a substack because the book went to the printer in the summer. So there's all kinds of new things that are happening. So in the sub stack, what I do is I fill in the new stuff that's happened and put it in the context of the book. It's brains and AI. What's happened is that Mistral and several other companies have discovered that if you use quality data, in other words, that's been curated or comes from a very good source and you may have to pay for it. And math data, for example, Wolfram Research. Even Wolfram, who founded Mathematica, has actually sold a lot of the math that they have. But with the quality data, it turns out that you get a much better language model. Much better in terms of being able to train with fewer words and a smaller network having performance that's equal or better. So that's same thing true as a humans. Right. I think what's going to happen is that the models will get smaller and they'll get better.

Another author to author question. I'll give you a negative example. So I believe back in the 70s, Kodak defined themselves as a chemical company. And we put chemicals on paper, chemicals on film. The irony is an engineer inside Kodak invented digital photography. But Kodak kept thinking, we're a chemical company. We're not a preservation of memories company. If they had repositioned their brains, they would have figured out we preserve memories. It's better to do it digitally than chemically. So now as an author, and you're also an author, I think what is my business? Is it chemicals, is it writing books? Or is it the dissemination of information? And if I zoom out, then I say it's dissemination of information. Why am I writing books? Why don't I train an LLM to distribute my knowledge instead of forcing people to read a book? So do you think they're going to be authors in the long run? Because a book is not that efficient a way to pass information.

Interesting. And this is already beginning to happen. So you know that you could train up an LLM to mimic the speech of people if you have enough data from them, movie stars. And also it turns out that you can not only mimic the voice, but someone fed in a lot of Jane Austen novels. I gave a little excerpt in the book. You can ask it for advice and it will start talking as if you're talking to Jane Austen from that era. And there's actually interesting, potentially Important way, if you have enough data about an individual, videos, writing and so forth. If that could all be downloaded. You're right. Into a large language model. It would. In some ways it would be you. Right. If it has captured all of the external things that you've said and done. So it might, who knows?

Terry, I have a company for you. There's a company called Delphi AI. And Delphi AI, you can control what goes into the LLM. So Kawasaki GPT.com is Delphi AI. And I put in all my books, all my blog posts, all my substacks, all my interviews, including this interview will go in shortly. Right. So you can go to Kawasaki GPT and you can ask me and 250 guests question. And I promise you that my LLM answers better than I do. And in fact, since you talked about Substack, every week Madison and I put out a Substack newsletter and the procedure is we go to Kawasaki GPT and we ask it a question like, what are the key elements of a great pitch for venture capital? And five seconds later we have a draft and we start with that draft. And I don't know how we would do that without Kawasaki GPT. So that ability to create an LLM for Terry is already here. And Delphi AI has this great feature that you can set the parameter so you can stay very strict. And very strict means it only uses the data you put in or can be creative and it can go out and get any kind of information. So, you know, if somebody came to Terry GPT and asked, how do I do wing suiting? If you had it set to strict, assuming you don't know anything about wingsuiting, it would say, this is not an area but my expertise. You're going to have to look someplace else, which is, you know, that's like better than a hallucination, right? You gotta try that. You.

I will, I will. I had no idea. And is this open to the public?

And I paid 99amonth for this. And you can set it so it subscribes to your sub stack, subscribes to your podcast. You can make a Google Drive folder and whenever you write something, you drop it in the drive and then it keeps. Keeps checking the drive every week and just keeps inputting. And I feel like I'm immortal. Terry, what can I say?

It really has a transformative potential for. Who would have guessed that this could even be possible a couple years ago? No one. I think it's really a transition.

But, you know, just before you get too excited by this I don't think that there is a market for people's clones because I'm pretty visible and I only get five to ten questions a day. It's a nice parlor trick. Oh, we can ask Guy what he thinks about everything. But after the first hour, six months later, are you going to remember there's Kawasaki GPT? I doubt it. So what you would probably go to is ChatGPT and say, what would Guy Kawasaki say are the key elements of a pitch for venture capital. And Chat GPT will give you an answer almost as good as Kawasaki GPT. And you'll never go back to my personal clone again.

Yeah, I think your children might someday.

Well, why would that be true? They don't ask me anything now.

Oh, what's interesting, a lot of times when, when someone dies, their offspring and close friends say, oh, I wish I had asked them that question.

Or.

It'S too late. It's too late. No, if they have something, you're there to ask the question.

Okay, I did it for my kids.

Up next on Remarkable People. You could push it around. It can do either. It has the world's literature that on both sides. And this is, this is exactly the problem is that it is reflecting you. It's a mirror hypothesis reflecting your kind of stance that you're taking. And it's very, in some ways, it has the ability like a chameleon. Right. It will change its color depending on how you're pushing it.

Meet Nicole Nicholas, Capital One business customer and co owner of Ansette Uncles, a plant based restaurant and community space in Brooklyn, New York that got its start from a need for unity.

The inspiration, it was born from the desire to create a space that felt like home, where we can connect community, culture, good food, and come together with family and friends. That's how we birthed aunts and uncles.

Nicole and her husband Mike were fulfilling their dream of bringing people together out of their home kitchen. But they soon learned that the demand for community was greater than they knew.

It became overwhelming and we were like, we need home, but not in our actual home. We realized that there was also a need in our community for something bigger in our neighborhood. So we had to find a place.

Moving from a home operation into a storefront was a huge next step. But Nicole and Mike were able to take it on with the help of Capital One Business.

It's not for the week. As a small business, finding resources is super important because that's the way you'll be able to manage and scale. We would have never done that without having Capital One to be able to help us along the way. The cashback rewards are very helpful. You know, it just gave us that Runway to be able to breathe a little bit. Then you get to focus on the cooking of the food and making the experience great.

To learn more, go to capitalone.com businesscards.

Thank you to all our regular podcast listeners. It's our pleasure and honor to make the show for you. If you find our show valuable, please do us a favor and subscribe, rate and review it. Even better, forward it to a friend. A big mahalo to you for doing this. Welcome back to Remarkable People with Guy Kawasaki. I'm gonna get a little bit political on you right now. It seems to me that people can try this who are listening. Go to ChatGPT and ask should we teach the history of slavery? Ask questions about should we have a biblically based curriculum in public schools? Go ask all those kind of questions. You're going to be amazed at the answers. So my question for you is, don't you think that in the very near future, red states or let's say a certain political party, they're going to block access to LLMs because if LLMs are telling you yes, we should teach the history of slavery.

Slavery.

I can't imagine Ron DeSantis is wanting people to ask chat GPT that question.

So now we're getting into hot water here.

You're tenured, right?

And it's not just chat gdp. We're talking about all of these high tech websites that repository of knowledge and information that you can search. They have a devil of a time trying to figure out should they have thousands of people actually doing this. They're constantly looking at, not hate things that are said on Twitter or whatever. That has to be scrubbed. Now the problem is who's scrubbing it? What, what do they consider bad and, and, and if humans can't agree, how can you possibly have a rule that is going to be good for everybody if there isn't any? I think it's an unsolved problem and I think it's reflecting more the, the disagreements that humans have than the fact that ChatGPT can't decide what to say. But it's interesting. You could probably push it in certain directions, right? I think that people have tried that, they've tried to break it one way or another.

It may be that many Republicans have never tried LLM, but I'm telling you, if they tried it, they would say LLMs are woke and we got to get all this woke stuff out of the System, I can't imagine.

Ah, ah, okay. My guess is that you'll, you'll get a W spoke person talking and coming to conclusion that this is flaming a conservative here. In other words, you can push it around. It can do either. It has the world's literature that on both sides. And this is exactly the problem is that it is reflecting you, it's a mirror hypothesis reflecting your kind of stance that you're taking. And it's very, in some ways it has the ability, like a chameleon. Right. It'll change its color depending on how you're pushing it.

That's no different than what people do in a conversation.

That's right. And also people are polite. They generally stay away from things that are controversial and yeah, we need that in order to be able to get along with each other. Right. It would be terrible if all we did was argue with each other.

About a year or two ago there was this, I won't prejudice your answer. There was this idea that we would have a six month kind of timeout while we figure out the implications of AI. Is that the stupidest thing you ever heard? How do you take a time out from AI? Let's just like time out and figure out what we're going to do.

That was done by I think 500 machine learning and AI people that decided that in their wisdom that we have to. You're right. It was a moratorium and I think it was specifically on these very large GPT models that we shouldn't try to train them beyond where they are because they might be super intelligent and they may have to actually take over the world and wipe out humans. This is all science fiction, right, that we're talking about. And in the book I, I came across an article, an economist, where they had super forecasters who had a track record of being able to make predictions about catastrophic events, wars and technologies, nuclear technology, better than the average person. And then they also compared the predictions with experts and it turns out that experts are a factor of 10 times more pessimistic in terms of whether something's going to happen or when it's going to happen than the super forecasters. And I think that's what's happening, is that they think that their technology is so dangerous that it's, it needs to be stopped.

When I read that section of your book, I had to read it about two or three times because it's exactly opposite of what I thought it would be, that super forecasters would be Armageddon and the technical people would say no it's okay. Like how do you explain that?

There's a simple explanation. I think though that everybody thinks that what they are doing is more important than it might be in terms of its impact. Actually, this is funny. When Obama was elected president, the local newspaper interviewed a lot of academics about, you know, he said he was going to support science and that was wonderful. And so the newspaper asked, what areas of science do you think the government should support? And almost every person said, but what I'm doing is the most important area to fund, you know, because they're the closest to it and of course they've committed their life to it.

So I, I mentioned that I had two absolute gems that I loved as quotes in your book. And I'm coming to the second one, and the second one is not necessarily a quote, but I want you to explain the situation when you say that Sam Altman had, shall I say, symptoms of Toxoplasma gondi, that the brain parasite that makes rodents unafraid of cats and more likely to be eaten. So why did you say that about Sam Altman?

Okay, so first of all, this is a biological thing that happens in the brain of the, of the poor mouse or rat. So there was a time when he would go to Washington and not just testify before Congress, but he would actually go and have dinners with Congress people and talk to them. And the history is that Bill Gates gets pulled in and he gets grilled in the congressional testimony and they have an aversion. So here's this guy who's going in and not just going for testimony, but actually going and trying to be part of their social life. So it just seemed that he was being contrary to the, the traditional way that most humans would deal with people who are out to regulate you. But actually somewhere later in the book, I, I, I think I identified another explanation which is that the regulation is an interesting thing because it basically puts up barriers, right? It turns out if you have lots of lawyers, you can find loopholes. There's always a loophole, right? And if you're rich, you can afford lawyers to find the loopholes for you. And of course the big corporations, high Tech, Google and OpenAI, they have the best lawyers, they can hire the best lawyers to get around any regulation, whereas some poor startup, they can't do that. So it'll give the big companies an advantage to have regulations out there.

Couldn't a scrappy small, under capitalized startup ask an LLM? What are the loopholes in this regulation? It would find them.

Ah, okay, well, so now you're saying that in fact, they could use their own because they're not going to be able to make their own LLM. They're going to have to use the other, the big ones that are already out there. And it could be that these companies are actually democratizing lawyers. By the way, it's not just lawyers and laws. It's also reporting. In other words, there's a tremendous amount of what they want to do is somehow if the companies have to have tests and lots of examples, they're going to require a lot of the FAA before you. The airplane is allowed to carry passengers. It's got to go through a whole series of tests and very stringent. It has to be put into the worst weather conditions to make sure it's stressed. Stress test. And again, all of that testing is basically for a large company. They have lots of resources to do that and it may not be easy for a small company. So it's complicated. But in any case, I think that what's happening right now is that. That the Europeans have this AI law that is 100 pages with very strict rules about what you can and cannot do. Like you can't use it for interviewing future employees for companies.

We just advocated for that.

Yeah, we'll see what happens in the US because right now it's not prescriptive, it's suggestive that we follow these rules.

And what would be the thinking that you can't use it to interview employees? Europe, what are they worried about? Oh, bias, bias, bias, as opposed to human bias. Like a male recruiter falls for an attractive female candidate.

Okay, that's also a bias. I guess there probably is some law there. I don't know. Not only are we biased, but we're biased in our biases, who we talk to, things like that.

All right, I gotta tell you one more part I really loved about your book is when you had the long description of legalese and then you had the LLM simplify a contract. And that was just beautiful. Like, why do terms of service have to be so absolutely impenetrable? And you showed an example of how it could be done so much better.

That that is happening right now. I think in a lot of places. That is. And this is a big transformation that's occurring within companies now. They are. The employees are using these tools in order to be able to help, first of all, keep track of meetings. You don't have to have someone there taking notes because the whole thing gets summarized at the end of the meeting. It's really good at that and speech recognition.

And then you also mentioned that when doctors are interviewing patients that instead of looking at the keyboard and the monitor, they should be just listening and let the recording take care of all that.

Right?

Yes, that's. That, that's a huge benefit because looking at the patient is. Carries a lot of information. Their expressions, the color of their skin, all of that is part of being a doctor. And if you're not looking at that, you're not really being a good doctor.

Okay, this is. This is seriously my last question. I love the fact that the first few chapters at the end, they had these questions that probably ChatGPT generated. Why didn't you continue that through the whole book? So every chapter ends with questions?

I don't know. I hadn't talked about it. I'll tell you. I wrote the book over a course of a year. And I think that it must have been the case that by the time I do use it throughout the book, I have sections and I actually set them apart and say, this is chat gdp. At the end, there's this little sign, open the eye sign. And I ask it to summarize parts. And at the beginning, I actually asked it to. Sometimes I asked it to come up with, say, five questions from this chapter. And that's where Alex the parrot popped out.

Am I. Am I the first person to catch the fact that Alex the parrot was not mentioned in the text? Except for the footnote?

You are the first person and I. I suspect there are others that notice that. But actually, it's good to have a few little morsels in there that you have a little detective story. Who is Alice?

All right, how about I give you, like, I really want you to sell a lot of copies of this book, so how about I give you, like, just unfettered. Give us your best shot promo for your book.

Everything you've always wanted to know about large language models and chat GPT. And we're not afraid to ask.

That's a good positioning. I like that. It's like that book way in my past, there was a book called everything you wanted to know about sex but was afraid to ask. Right.

There's a takeoff bald, you know, ripoff.

As I learned from five jobs. You gotta learn what to steal. That's a talent in and of itself.

You're paying homage to the past. But I'll tell you, I wrote this for the public. I thought that the news articles were misleading and all this talk about super intelligence was. Although it's a concern, it's not an immediate concern, but we have to be careful, that's for sure. And it helps. I'm trying to help people. When I give talks, they ask, will I lose my job? And I say, you may not lose your job, but it's going to change. And you have to have new skills. And maybe that's part of going to be part of your new job, is to use these, this, these AI tools.

Well, as you mentioned in your book, when we started getting farm equipment, there are a lot less farmers. You could manage thousands of acres with one person, right?

Yes, that's true, that's true. But those, the children went to the cities and they worked in factories and so they had a different job, but it wasn't working to get food. It's working to make cloth automobiles and things and LLMs.

Eventually.

Yes, eventually for some of us.

I just want to thank you, Terry, very much. I found your book very, very not only interesting and informative, it was just. There were places where I was just busting out laughing and I'm not sure that was your intention, but when I read that thing about Sam Altman's brain has that thing that make rodents less afraid of cats, I'm like, oh my God. God, this guy is a funny guy.

So. So I thought I'd make it entertaining so that people can appreciate in some way we're too, we're just people serious about this. Let's have some fun.

One of my theories in life is that a sense of humor is a sign of intelligence.

Actually, I'll tell you if this is interesting. You know who gets the Academy Awards? It's the actor who's in some terrible drama where something bad happens and so forth, and then they overlook all the fantastic comedians. It turns out it's much more difficult to be a comedian than to be somebody who's, you know, has angst and they're not, they don't. They're not given the same respect. I had no idea that you've read the whole book because most of the people who interview me, they know some, they've read some parts. But you, it sounds like you know the whole book.

I could. Do, you know, the story of the chauffeur and the physicist? Physicist, okay. This is along the lines of what you just said, that I read the whole book. So this physicist is on a book tour. Let's say it's Stephen Wolfram or Neil Degrasse Tyson. So anyway, they're on this book tour and they're gonna make four stops in this cities and they, the chauffeur takes them from stop to stop. So the chauffeur sits in the back and listens to the first three times the physicist gives the talk. At the fourth time the physicist says I am exhausted. You heard me give this talk three times. You go give the talk. And the chauffeur says yeah, I can do it. I heard you three times. Chauffeur goes up, gives the talk but he ends early. And so the mc, the host of the event says to the chauffeur, oh, we're lucky we ended early. We're going to take some Q and A from the audience. So the first question comes up and it's about physics and the chauffeur has no idea and he says this question is so simplistic I'm going to let my chauffeur sitting in the back answer. So I'm your chauffeur.

Oh, that's wonderful.

All right, Terry, thank you. I truly enjoy this. All right. All the best. This is remarkable, people.