EP #35 AI & GPT-4 Revolutionize Education w/ Sal Khan

I'm 32, juggling family, working full time and earning a bachelor's degree at University of Phoenix. I earn career relevant skills with every five week course. Skills I can use now, not just after graduation.

Earn skills in weeks, not years. Visit Phoenix.

Edu Some people think nature is like this, but actually it's like this. That's why Columbia engineers everything we make for anything nature can throw at you. Colombia engineered for whatever.

There's a young girl in Afghanistan. Her Name's Sultana. About 12 years ago, Taliban takes over her town, didn't let her go to school. She gets on Khan Academy. She self educates. She ended up lying to her parents, going to Pakistan to take the SAT because it's not offered in Afghanistan. I'm like, this young woman's amazing. I tried to figure out a way to forget her visa. Luckily Nicholas Kristof, New York Times found out about her, wrote an op ed meet Sultanah, the Taliban's worst fear. She had no transcript, no grades. All she had was like eight years on Khan Academy and an SAT score. And Arizona State University admitted her credit to Arizona State University. Sultana now is a member of the research faculty at Tufts. In quantum computing. You can imagine how many more Sultanas we can have if we have an AI supporting them. We've always aspired at Khan Academy to be, hey, every learner on the planet can essentially get a tutor. Every teacher could get a teaching assistant. OpenAI reached out to us and they said, hey, we're working on our next generation model, which we all now know as GPT4. And when we saw this technology, which now folks are Familiar with the GPT4 technology, we said, wow, this might actually do the trick.

It's extraordinary and it really is about personalizing every student's education.

The gold standard in education has always been personalized tutoring. I think this passes the Turing test in a lot of the tutoring use cases. It constructs its own private thoughts on how it would approach the problem that the student does not see. We think this is going to be both exciting and scary for folks. I would say 95% of what you just described is not a research effort anymore. It's just an engineering effort.

Hey, everybody, it's Nick. I'm Peter's producer. We just got back from a 360. It was a phenomenal event. Peter actually sat down with Sal Khan, and if you're unfamiliar with Sal, he's the founder of Khan Academy. And Khan Academy is literally revolutionizing education in every way imaginable. Peter and Sal sat down to discuss the future of education, not just in America, but in the world. We all know there's an education crisis and Peter and Sal really dissected how artificial intelligence and machine learning is and can be used to solve these challenges within education over course of the next decade. And if you're interested in understanding how we're going to solve the American and global education crisis, this is the podcast for you. This podcast was taken as an excerpt from A360, which is Peter's private community. If you're interested in learning more, we've included the link in the bio. Enjoy the episode.

Saul, thank you for coming.

That was very energetic.

Yeah, yeah, they have an abundance of love for you.

Yeah.

It was great to see you. A few months ago and Sol was playing chess against champions for a fundraiser. It was a fun event.

It was humbling. Yeah, it was the equivalent of like doubles tennis, but on chess. So it was like Silicon Valley civilian, plus a grandmaster plus another Silicon Valley civilian and another grand. Yeah, it was humbling. I lost to Magnus Carlsen and Yuri Milner's 8 year old daughter.

I know, I know. It was a party Yuri Milner put on and it was, it was amazing. And you were losing in front of Sergey and Eric Schmidt and the entire Google team and every billionaire in Silicon Valley. Humbling. Yes. It's been a busy week for you.

Yes, it has been.

But it's like, do you feel like you just put the roadster into fifth gear or something?

Yeah. And just for context of what Peter's alluding to, and it feels good to talk about it because I was under NDA for six months. But about six months ago, late July, early August, OpenAI reached out to us and they said, hey, we're working on our next generation model, which we all now know is GPT4. And they said, hey, we want to talk to Khan Academy for two reasons. One is we want GPT4 to be truly good at knowledge and we're going to benchmark it by things like the LSAT and AP exams. And you all have tens of thousands of items. Can we use them to train, evaluate? Yeah, maybe. But the second one is they said, we think this is going to be both exciting and scary for folks and we want to launch with a handful of organizations that can show a social, positive use case. And so we're like, okay, whatever. And I was familiar with GPT2, GPT3. I was like, yeah, I don't know how good this is going to be.

You've known Sam for some time.

For a little bit. For a little bit. But then two weeks later, they showed us the first run, this was early August, and showed us an AP bio question and said, sal, what's the answer? He's like, okay, I think the answer is C. And then it was able to say the answer C. I'm like, well, ask it why? And it was able to explain it. And you have to realize this was three months before ChatGPT. When ChatGPT came up, I emailed Sam and Greg and Craig Brockman, and I'm like, I thought we were going to wait until March to launch this thing. And they're like, no, we didn't launch anything. This is just a chat app on top of stuff we've already launched. But people are excited about it. But you could imagine when in August, we were seeing something that was far better than ChatGPT, it was like, oh, wow, this could do a lot. And we've always aspired at Khan Academy to be, hey, every learner on the planet can essentially get a tutor. Every teacher could get a teaching assistant. And when we saw this technology, which now folks are Familiar with the GPT4 technology, we said, wow, this might actually do the trick. It was even good at math.

This episode is brought to you by Levels. One of the most important things that.

I do to try and maintain, maintain my peak vitality and longevity is to monitor my blood glucose. More importantly, the foods that I eat and how they peak the glucose levels in my blood. Now, glucose is the fuel that powers your brain. It's really important. High, prolonged levels of glucose, what's called hyperglycemia, leads to everything from heart disease to Alzheimer's to sexual dysfunction to diabetes. And it's not good. The challenge is all of us are different. All of us respond to different foods in different ways. Like, for me, if I eat bananas, it spikes my blood glucose. If I eat grapes, it doesn't. If I eat bread by itself, I get this prolonged spike in my blood glucose levels. But if I dip that bread in olive oil, it blunts it. And these are things that I've learned from wearing a continuous glucose monitor and using the Levels app. So Levels is a company that helps you in analyzing what's going on in your body. It's continuous monitoring 24 7. I wear it all the time. Really helps me to stay on top of the food I eat, remain conscious of the food that I eat, and to understand which foods affect me based upon my physiology and my genetics. You know, on this podcast, I only recommend products and services that I use that I use not only for myself and my friends and my family that I think are high quality and safe and really impact a person's life.

So check it out.

Levels link Peter Give you two additional months of membership. And it's something that I think everyone should be doing. Eventually. This stuff is going to be in your body, on your body, part of our future of medicine today. It's a product that I think I'm going to be using for the years ahead and hope you'll consider as well.

It's extraordinary and it really is about personalizing every student's education in an extraordinary fashion.

No 100%. I've always said, and this was before the AI stuff, that the gold standard in education has always been personalized tutoring. If you were to go back 2,300 years, not many people got an education, but the ones that did got a pretty good one. If you were Alexander the Great, Aristotle, whose mentor's mentor was literally Socrates, like, he knew how to do a Socratic conversation, which is the inspiration for a lot of what you just saw there. But then two, 300 years ago, when we had mass public education, which was a major innovation for humanity, but to do it economically, we had to borrow the ideas of the Industrial Revolution, which is batch students together, move them at a set pace and look, that got a lot of us to where we are now. But we know that a lot of folks weren't able to make it through because you get an 80%, 70%. Too bad the factory line keeps moving. What's exciting now is, and there's been tons of FB studies about personalized tutoring with a high quality tutor, but it's just never been economic. And Khan Academy pre AI was what was. Well, I started as a tutor with my cousins and I saw what was happening with them. And then early Khan Academy was my attempt to scale that. And then we have actually, we created another nonprofit called Schoolhouse World, which does peer to peer tutoring. But what the AI does is it's the scale of Khan Academy. But you can essentially give the. Those weren't made up interactions that we just said, oh, let's just dream. Those were actual screenshots of Khanmigo working. So, I mean, some of the things for those of you who haven't played around with how GPT4 is different than ChatGPT or GPT 3.5, those are very nuanced conversations. That's the AI acting like a really good Socratic jury. I think this passes the Turing test in a lot of the tutoring use cases.

Extraordinary. This is amazing. And it's a hint of what is possible. And this is a interaction over typing. I guess we can add a voice to text and text to voice version of that, but a lot of people are experiential learners. How far are we from what I'm hoping we'll see, which is I enter into a metaverse, into a virtual world, and Aristotle is sitting on a piece.

Of marble over there.

And that AI of that NPC is all knowledge about Aristotle. So when I speak to Aristotle, it's as if I'm speaking to Aristotle and I say, can you show me around and tour me? And I learn about ancient Greece through that conversation.

Yeah, I mean, I would say 95% of what you just described is not a research effort anymore. It's just an engineering effort. Someone just has to put the pieces in place. Aristotle is one of the folks you can talk to already on Khanmigo, and you can have a very meaningful conversation. The way we see it evolving. And every time we think about it for 10 more minutes, we're like, oh, maybe we're being too narrow. We could also do this, we could also do that. But I think speech to text, text to speech, that's easy. That's easy. And it's going to come literally in months. And it's amazingly good. It's almost once again hard to differentiate it from a real human being. Then you're going to have, I think, a layer of memory. You know, right now, every interaction that you have with, if you're a student, it only knows about the current conversation.

Yes.

It doesn't know about your last conversation.

It doesn't remember your favorite sports star.

Exactly. But that's just an engineering thing. And we've already started making it pass variables to a persistent store and back. I mean, there's some interesting things that we did to make the math work. People will find that Khanmigo, even though GPT4 is available to a lot, that the math works a lot better on Khanmigo. And I'll tell you, one of the hacks we did and when we realized it's not even really a hack, is that if you just ask it the solution to a problem, it's pretty good. It can say, here are the steps. But then when a student makes an error, let's say they don't distribute a number on both all the terms in a parentheses, it doesn't always identify. Sometimes it'll say, correct, good job. But then it'll. But then it'll give the right answer. And it's like, wait, it wasn't a good job. You should say, hey, it looks like you made a mistake. And what we realized is if you were a tutor and if I just said, here's what I did, start talking. And you weren't allowed to think before you were to talk, your first word might be, good job. And then you start, like, the next words just come out. That's how a large language model works. Well, we said, well, no, we worked with the open eye, folks. Well, what if we gave it a chance to have its own private thoughts first? And so what we do is essentially it constructs its own private thoughts on how it would approach the problem that the student does not see. And then. And in those private thoughts, I mean, it's funny because we can read these private thoughts even though we told them we couldn't, it'll say, well, I think the student might have made a mistake here, because this is how I did it. And then when we make it, verbally compare the student's results to it, the math accuracy goes up by three or four fold.

Amazing.

And so there's this idea of like, think before you speak. And even though at first we're like, it's a hack, but then we're like, no, that's actually how we operate. So, yeah, I think you give it memory. I think it's an engineering effort to put it behind some type of, you know, if you wanted to put it in VR or whatever. But I'm also really interested in it facilitating more human to human interaction. So one of the things that I've been, you know, I'm hoping in the next year we can have it facilitate. We have a debate where you can debate with the AI and listen, I.

Think debate is one of the highest forms of learning and underutilized asset that we humans have in school. One of the thoughts, my kids love the game Averse, right from Roblox. And I'm respectful of the game designers who do an amazing job. In the education world, everyone starts with 100% score. And every time you get something wrong, your score goes down. And the gaming world, every time you start with zero and sometimes you do something right, your score goes up. That gamification hits very fundamental neural processes. Are we going to start to see that level of gamification coming in? I saw a little bit of it here.

Yeah. And this is something that obviously we were attempting to do pre AI as well. I will say education is the one thing because we have the games get to change the Objective function to optimize around the game mechanics and the dopamine hits and all that. In education, we're like, no, the kids really learn how to factor a polynomial. How do we do it? But you're absolutely right. I think this is only going to make it more, there's more opportunity for positive gamification.

Amazing. So let's talk about the scale of Khan Academy. What it's been, I mean, just the number I have here that before the pandemic you had nine public school districts involved and then afterwards was 500 public school districts. Is that an accurate assessment? What's been the impact of Khan Academy so far? Number students, teachers and such?

Oh yeah, the very high level numbers. 150 million registered users, something like that, and then give it up.

150 million registered users.

No, you know, that's a little bit kind of a vanity metric, but anyway, but there's, there's, it's, you know, depending at the peak of the pandemic, we had, you know, 30 million students coming a month, but it's about, it averages about 10 to 20 million. The pandemic we did see a threefold increase. Pre pandemic, we had about 30 million learning minutes per day on Khan Academy. That grew up to about 90 million. It's in 50 plus languages. In terms of the impact, there's a lot of ways you can measure the impact. You could say, okay, how many, you know, it's on the order of 12 billion minutes per year and, you know, what would be the economic equivalent of that? But we've looked at some of our school pilots and we pretty consistently see that if students are able to spend even 60 minutes a week, 30 to 60 minutes a week, they're accelerating 30 to 50% over their peers. And you could, you know, the economic value of that is pretty high if you think about how much we spend on education. And so what we've been trying to do, the schools you mentioned, we've always had a ton of teachers and schools use Khan Academy. But five years ago we started going to a bunch of districts. We're like, look, a lot of the teachers are already using us. How do we get used systematically or systemically in your district? And the districts almost always say, oh, well, yeah, thanks, my niece got through the SAT and this and that because of Khan Academy. But for us to use it in that way, you got to give us support, you got to give us training, you got to give us integration with our rostering system, all this enterprise level stuff. So we started doing it. We're like, look, if we're serious about our mission's free world class education for anyone, anywhere, we genuinely think we can move the dial for nations for the world. Eventually we've got to do this because that's how we get integrated. So that's where we went from nine school districts to arguably almost every school district definitely in the US and much of the world is already using us in some way, shape or form. But those nine, which are now about 500, those are ones that are formally working with us. We're doing all of this type of stuff.

So what is your. Let's give it up for that. Yes. So what's your moonshot? Where do you want to go next? How do you want to scale the use of this AI? What's your. Where do you want to go?

Yeah, well, you know, the goal has always been all of the core academic material from pre K through the core of college, provided in a way that's personalized, master based. Try to emulate what Aristotle did for young Alexander. And then the other is we're going.

To have war mongers all over the place.

Hopefully not that, but hopefully the well skilled people at whatever they do. And then this is what most people don't associate with this right now, but we're actively working on is that how does that work on Khan Academy translate into credit and eventually jobs and opportunity. And we're starting to work on that. What I think the AI does, even pre AI there were certain, I'm guessing a lot of folks when we were young could have gone on a Khan Academy and just gotten obsessed with it. And we have a lot of kids around the world who are already doing there's a young girl in Afghanistan, her Name's Sultana. About 12 years ago, Taliban takes over a town, didn't let her go to school. She gets on Khan Academy, she self educates. For her, the big aha was when she realized she was learning more than her brothers in the Taliban control school. And I'm like, not a high standard. But she ended up lying to her parents, going to Pakistan to take the SAT because it's not offered in Afghanistan. And that's when I found out about her. She met some people on the Internet. I'm like, this young woman's amazing. I tried to figure out a way to forget her visa. Luckily Nicholas Kristof, New York Times found out about her, wrote an op ed Meet Sultanah. The Taliban's worst fear. That got her and credit to Arizona State University because she had no transcript, no grades, all she had was like eight years on Khan Academy and an SAT score. And Arizona State University admitted her because she wanted to be a physicist. And Sultana now, who I now know quite well, oh, there's more, there's more. That's like an infomercial. Sultana now is a member of the research faculty at Tufts in quantum computing. So, yeah, finish the thought though. There are sultanas in the world, but you could imagine how many more sultanas we can have if we have an AI supporting them. Sultana could just power through.

Yes.

Hey, everybody, this is Peter. A quick break from the episode. I'm a firm believer that science and technology and how entrepreneurs can change the world is the only real news out there worth consuming. I don't watch the Crisis News Network. I call CNN or Fox and hear every devastating piece of news on the planet. I spend my time training my neural net the way I see the world by looking at the incredible breakthroughs in science and technology, how entrepreneurs are solving the world's grand challenges, what the breakthroughs are in longevity, how exponential technologies are transforming our world. So twice a week I put out a blog. One blog is looking at the future of longevity, age reversal, biotech, increasing your health span. The other blog looks at exponential technologies. AI, 3D printing, synthetic biology, AR.VR blockchain. These technologies are transforming what you as an entrepreneur can do. If this is the kind of news you want to learn about and shape your neural nets with, go to dmandis.com blog and learn more. Now back to the episode.

So the question is what changing role of the teacher should be today and how do we train teachers to embrace rather than fear the new world of education? Teachers, heroes of society, underappreciated, underpaid. Talk to me about what you're trying to do for the teaching community.

Yeah, I think the big there will be no. You know, when people think, I mean, the whole topic today is abundance. When people talk about what we're going to do with all this abundance, I think there's never going to be like more human connection than what we know what to do with. And I think that's what fundamentally the role of the teacher is going to be optimized. And I think that's why most teachers become teachers, to become connected to their students. So even pre AI, we're like, look, the role of the teacher does not have to be give the lecture, they shouldn't be grading things, et cetera, et cetera. Students can learn at their own pace and they can do workshops, do more Socratic dialogue. They can do labs, they can do more experiential learning, take the kids on field trips or just tutor them while other kids are able to use the technology to learn at their own pace. The AI, I think just up levels that even more now it can work in every subject and every language. Every language. And it's not just what traditional Khan Academy could do, which is like traditionally machine gradable type of exercises where you can go pretty high level, but now you can have students write free form and it can grade for you and all that. So I think the teacher of the future is really going to make their classroom an amazing experience like truly experiential learning where they might use the AI. Hey everyone, we're going to talk to Zeus this morning. Let's get your questions together. Special guest or AI. And I really see for the teacher, the AI becomes like an army of teaching assistants which every teacher would love to have, which is like AI. I want you to group these 30 kids, pair them up into pairs and have them debate each other on this topic and then give me a summary of how they did and who was engaged and et cetera, and how I could work with them. I actually think we can talk about all the different areas where AI is going to be interesting, but I think it's going to be most fun in education and probably in teaching if it's used well. And you saw in the Khanmigo, one of the first things we're launching with is tools for teachers. Tools for teachers. I think the age of static. I've always talked about personalized curricula, but the age of static curricula is dead. Teachers are going to be able to. You could take something you already have and say, oh, my teacher team just won the super bowl last night. Make it about that. Or there was a SpaceX launch. Make it about that. Or my kids are into this. You know when you ask Khanmigo, if a student says, why do I need to learn this? It says like, well, what are you interested in? And then the student and they have a conversation and says, well, you know, if you want to be a soccer player. And it's honest with kids too. It's like, all right, you might not use this if you're playing, but if you want to go to NCAA scholarship or this, that you might still need to do this. So like, it all also doesn't BS the students, which kids appreciate. But I think that's going to be the fun of the teacher, which is be the conductor of the orchestra.

Amazing. Amazing. You're going to come back for questions after Jacqueline speaks. Give it up for Sala Khan, everybody.