BoulderTalks Dr. David Joyner: The Evolving Roles of ChatGPT & Generative AI in Education

Thank you for joining us today. This is actually technically our third series in our inaugural Boulder Talks Speaker Series. This is our third talk. Unfortunately Yakut wasn't able to join us for the second one because she got Covid on the flight out. But she will be coming back soon and we'll let you guys know. But this talk does wrap up our series this year. This series, as I said before, really comes about from a desire to bring to campus visiting experts who are innovating quality online education through this community that we hope to build. We're hoping to explore topics of learning excellence, innovating credentials, expanding access and opportunity, and ensuring success across various modalities. And with so much of what we do in online education, this is already a community. So I do want to recognize our co sponsors for this event the center for Teaching and Learning, the Office of Information Technology, Continuing Education, University Libraries, Office of Digital Education and the College of Engineering and Applied Sciences, and the Office of Academic and Learning Innovation. So without further ado, I'm going to introduce our speaker today and I'm definitely going to use my notes for this.

One because it's a good long one.

David Joiner is Executive Director of Online Education and the Online Master of Science in Computer Science, otherwise known as the omscs, at Georgia Tech's College of Computing. His research focuses on online education and learning at scale, especially as they intersect with four credit offerings at the graduate and undergraduate levels. His emphasis is on designing learning experiences that leverage the opportunities of online learning to to compensate for the loss of synchronous collocated class time. This includes leveraging artificial intelligence for student support and assignment evaluation, facilitating student communities in large online classes, and investigating strategies for maintainable and interactive presentation of online instructional material. As part of his work, David teaches online versions of educational technology, human computer interaction, knowledge based AI, and Introduction to Computing. He's also the Vice Chair of the Steering Committee for the ACM Learning at Scale Conference, as well as the General chair for the 2019 and 2020 conferences. David has received several awards for his work in teaching online, including the 2019 USG Regents Teaching Excellence Award for Online Teaching, 2018 Georgia Tech center for Teaching and Learning Curriculum Innovation award, and the 2016 Georgia Tech College of Computing Lockheed Excellence in Teach Award. So with that, I'd like to welcome David up and have him give us this wonderful talk.

Thank you so much for powering for my way too long introduction and very short a version of that. So hi everybody. Like Michael said, my name is David Joyner. I work at Georgia Tech. Most of what I do is run our online Master of Science in Computer Science program. So we get to the end. Feel free to. Feel free to ask any questions about that as well. But the past few years, especially in the wake of ChatGPT and the fact that my background is really in AI and education, I've seen more and more of my conversations drift in this direction. And so it's nice to be able to kind of bring this to the world for a very long time. Whenever we talk about AI in education, we kind of get kind of glossy and kind of glazed over looks of. I don't really understand what you're talking about. That's not something I can immediately apply in any of my classes. And so it's great that you're working on that, but, you know, it's not for me. And the past couple years have been surreal because suddenly everybody's interested in it and interested in some ways, terrified of it in other ways. I spent a lot of time talking, especially to K12 teachers who are being forced to adjust to this far more rapidly. It's been interesting to see the diversity of reactions to something this kind of seismic. It's interesting, I feel like some of these recent changes have been seismic in a way that we haven't seen very often, at least not in my lifespan. We haven't seen sudden seismic changes like this. It's kind of this, and I mean, Covid's forcing us into online learning are the two things I can think of that something happened really, really fast forcing us to change aspects of what we do day to day. What was interesting about it, like the title of this talk suggests, is that those roles are kind of evolving and changing over time and trying to look at kind of how things are going to change in the future. I talked to one teacher who, in the wake of ChatGPT coming out in November 2022, redesigned all her assessments very carefully, going back and forth chatgpt to see what its weaknesses were and what its strengths were, only to see all that work completely disappear when they updated the underlying model a few months later. And so really trying to look at what kind of changes we can see coming down the pipe and how we can adjust kind of a more durable way against things. I like to start this, though, and this kind of comes from some of the early talks where people were really kind of panicked about some of these things. I think it's been a good thing to keep because now it's kind of putting things in perspective in a different direction by looking at some of the other historical. I say threats to education. I say threats because a lot of the early conversation around some of these new laws, language model based artificial intelligence were really about this idea of why even bother teaching anymore? AI can do it all, it's going to write all the students essays, it's going to do other problems. Why even bother anymore? Which is interestingly enough, a conversation that's happened over and over throughout human history. Calculators are the one we hear referenced most often. There are protests about using calculators too early, which granted, I think these are justifiable because this is about using at the lower school level. I am an elementary school daughter and yeah, it seems odd the idea of giving her a calculator instead of teaching her addition and subtraction by hand. But I think we also know that the idea of calculators coming along and wholesale replacing massive portions of math curricula did not come to fruition. In fact, it became a foundational part of what we teach in math. And now teaching how to use a calculator effectively is a key learning goal for most math classes. Before that I found this kind of interesting doing this initial research about this, that when printed news media first started to catch on, there was this view that this push towards printed news media was going to really isolate people. Because before that news was delivered via a pulpit kind of like this. And if people were getting it via their own little printed newspapers, that was going to radically change the way people got their information in a way that has some implications for education as well. Before that it was this idea that books are going to be problematic, that an overabundance of books would cause an overabundance of content would be confusing and harmful to regular people. Because regular people, they weren't ready to consume this volume of books. That's for the early learned scholars. And so it would be disastrous to unleash this on the students of the world, which I don't think it was. Or going even further back, my favorite one was, and this, I should say this is a quote attributed to Socrates by Plato, which probably means Socrates never said it, but Plato wrote it in. The character of Socrates in one of his dialogues was that writing would create forgetfulness in learners souls because they might use their memory. So the very fundamental idea of learning to read and write would be a bang to education because students would learn to do that and wouldn't bother remembering things anymore because they wouldn't need to anymore, because now it's written down why bothering remembering something that's not written down. This one I actually think has the most clear analogy to some of the AI stuff we see nowadays. Because the thing I think we're pushing back on a lot is the idea of some students thinking, well, why bother learning it if the AI is going to know it? Whenever I need to know it, I can just go look it up when I need it and it'll be readily accessible right there. So this is kind of similar to that. But I think we all can kind of agree that none of these kind of things did away with education. One of my favorite examples of this comes from calculators, again, which was There was an 11 year period at the beginning of which the state of Connecticut thought that calculators were so fundamental and so required, so ubiquitous that they began requiring calculators on statewide Math tests. And 11 years later, the state of California prohibited calculators on statewide math tests because of perceived problems with what they meant for what students were learning. That so you have this kind of really common technology that for a full decade was kind of in a weird enough space that different people were erring very heavily on one side or the other about whether or not to require them, whether or not to forbid them, whether or not to expect them, anything like that. And that happened over a decade kind of period, which is something that we'll talk about in a second. But there's other things that, you know, historically done the same kind of thing. Search engines fundamentally change the way we do information lookup that many of us may remember learning how to navigate the Dewey Decimal system in the library in order to go find the book that had the information that we needed. I don't think we teach that anymore. Word processor. I think we underestimate how much this fundamentally changed the writing process because no longer was revising an essay about getting all your feedback and writing a completely fresh version of something from scratch. It became much more iterative, much more fine grained, and we expected students to do more with them. And smartphones I always like, because smartphones are kind of the most recent, I think comparable type of motivation to this. And I remember talking to a teacher in actually recently, but she was recalling the story from the late 2000s about the first time a student brought an iPhone to class. And her observation that when every student had a technology like that, what she could do in her class that she couldn't do right now, because right now it was all whether or not how many computers were accessible and how much access was accessible. And the idea that if everyone had that, there were avenues that would open up that were completely not yet available to her and kind of see in the future of once this catches on, this is going to radically change what I can do in class now. I share this because I've given. I've had conversations like this a few times in the past and most people, I think, feel like GPT is a bit more significant than many of these. And I question why that is. And I think when I ask the question of why does GPT feel more significant to people? The answer I usually get is that it's more like a person. I like this example because this is one of the more human like dialogues I've gotten out of ChatGPT. So I asked it essentially in 50 words, tell me about David Joyner. And ChatGPT responded. David Joyner is an American actor and educator best known for portraying the children's character Barney the dinosaur in the TV series Barney and Friends from 1991 to 2001. Which is a completely true statement and is not true about me, but is true about the far more famous Barney and dinosaur or far more famous David Joyner that did play Barney and Dinosaur over those years, which elementary school me was extremely excited to see in the credits of. Of Barney. But so it gave me a description of the actor David Joyner. And so no, no. What about other David Joyner's? Do you have any others? And chatgpt says, no, I've never heard of any other David Joyner. So what about David Joyner from Georgia Tech? He says, oh, David Joyner from Georgia Tech. Well, he does this and he looks at online education, runs the OMSCS program. And I say that this is a very human like interaction because you can kind of imagine having this kind of dialogue with someone in real life. You know, you see someone. Do you remember, you know, Elliot? Elliot? No, I don't remember Elliot. Yeah, Elliot. Elliot Smith. He was your accountant back then. Oh, that Elliot. I remember that person. How is he doing? It's about giving that level of context that can call the mind and memory of whatever you're talking about. It's a very human like kind of thing, which is not terribly surprising knowing how GPT works under the hood, that it works very much like a human brain. So that's the most common response I get to why GPT feels more significant than some of these other things is it acts more like human, at least proficiently. I question the extent to which that's true because I wonder. I won't nep at these, but I wonder to what Extent we would have said the exact same thing about certain earlier innovations. So like calculators, for example. When calculators came along, the calculator or the human was the only person who could do, you know, graph an equation and things like that. So that was a very human like behavior at the time, but it quickly became not a very human like behavior. So is this just a recency thing? And now there are a couple quotes I like to echo this. Bill Gates said we always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next 10 years, which I think we're about a year and a half away from when ChatGPT first landed kind of kicked off this again, renewed interest in AI among the public. I think we're kind of seeing that the world hasn't changed quite as much in the past two years as we probably thought it might when this first came out. But we're going to see a lot more change coming down in the future. And Roy Amara have a quote that's about the same kind of thing, but it's about the effect of technology in the short term and long run. The short term effect is much less than we expect, but the long term effect is much, much greater. And so I think we're starting to see that some of this is this, that as we've gone along interacting with these technologies a bit more, we've started to see kind of our expectations kind of hedge a little bit. We started to learn what it's good at, what it's bad at, what kind of mistakes these technologies make and things like that. The other thing that always strikes me is one of the main reasons why this appears to be so significant is just the rate with which it landed everywhere. So I looked up the rate with which other technologies reached everybody. The telephone, for example, took this is measured in months. So 50 years, if I'm doing the math right, to reach 50 million users. Television took a decade, personal computers took over a decade. Cell phones you see dropping. But even the iPhone, like I said, the teacher I was speaking to who mentioned it at the observation of what's it going to be like when every teacher or every student has one of these in my classroom, it's going to change everything. It was three years before it reached 50 million users. And most of those probably were not high school students at the time. So we had a lot for all these technologies. I think probably for graphic calculator. That's one I really like to find for all these Technologies, we had a very long lead in time to start to observe and start to see, okay, that one student who brought a graphing tracker to the school. What were they able to do? Okay, once every student has this, how do I need to change things? So we had some time to adjust what we expect based on these technologies rolling out very slowly. ChatGPT. I can't actually find how long it took to reach 50 million users. I know it took two months to reach 100 million users, and so I'm guessing there was a kind of tapering off. So it's probably less than one month to reach 50 million, but it kind of landed everywhere all at once. In fact, I've got a book coming out in a couple months about AI education. And my initial working title that my publisher rejected was Chatbots Everywhere all at Once. Not because chatbots suddenly really did become a thing, but that's kind of how it felt back in November 2022. Suddenly it was everywhere. Everyone had access to it in a way that hasn't happened with prior technologies. It's as if we go back to the calculator analogy. It's as if everyone had a TI83 just appear on their bedside table one night and brought him to school the next day and then brought it home to do their math and work the next day. As if we had to adjust kind of that quickly for comparison. I like to look at other technologies that also took off that fast. Pokemon Go, you know, would be one Angry Birds. Does anyone remember Draw Something? No one does. Draw Something was a huge thing in, like, 2014 when I was researching for this talk, I came across it as. It's mentioned in a lot of these things as an example, and it took on way faster than other things I included just as along with Google as a sign of quickness. To take off is also not necessarily a sign of longevity. Google reached 50 million users in three months and shut down about five years later. And I'm convinced that when it shut down, most of its users were actually our students, because for whatever reason, students in my program really loved Google there at the end. So I'm pretty sure most of it was just us. And so I kind of use it now. I don't think ChatGPT is going to go the way of Draw Something or Google unless OpenAI, you know, really steps in it in the next few months. But even if it does, you know, one of the others is going to come along. So I think this is going to be obviously more durable. And even with things like Google Those were obviously technologies that peaked very quickly, but the domain that they represented was persistent over a much longer period of time. So I tend to think this is one of the main reasons why we consider some of these new innovations to be so much more significant to those early technologies is just because the rate at which they forced us to adjust. It kind of goes back into what I see as a cycle around how teaching adjusts to new technologies over time. The teachers teach a skill, teach some skills, have some curriculum, some lesson plan, and then something comes along, some technology comes along, it performs that skill on behalf of students and reduces the need for students to learn that skill on its own. And then I panic in quotation marks because of the way the media very often portrays it. But teachers panic, panic about what? How to teach. Now we can see some of these other articles that have come out about teachers in a panic over going back to school would fall when ChatGPT is so common, everyone using it to cheat and everything like that. My experience at least has been about 10% of teachers I talk to do panic, and about 50% kind of are ambivalent, and about 40% are really excited about what it means for what they can do. But the general idea is that there's some thought about what do we teach now and how do we teach it now. Now there's this new technology that's very heavily available, and then we see a kind of a revision process go into it. The teachers revise their curriculum based on this new technology. But this calling, this cycle is not really the right thing, because what happens is we don't land back where we started. We land back on some new higher order skills that we teach now, now that we have access to some of this technology to enhance what we're able to do. And this cycle is repeated with over and over throughout the history of the interplay between education and technology. We very often see these new technologies come along, like graphic calculators are a classic example in education that really fundamentally changes what we can and should expect students to do first, because it takes some things that we may have had students do by hand and makes it so they can do it much more quickly with access to this technology. But as a result, we can then take them further than they were able to go otherwise. I put what and how in two different ways, because I think those two represent two different kind of revisions that go in and what happens first and what happens second as they have to do the what very often I think is the thing that confronts us from the outset Is that especially with things like ChatGPT and calculators, we see it doing the exact things that we've assigned to students initially. And that's what forces this kind of panic view of why have students do that when now they have technology that can do that. And we revise what we teach in a few different ways. There are some skills that are permanently removed. There are some skills that are recontextualized, and most importantly, there are some entirely new skills that we end up unlocking. Old skills are permanently removed. I always hesitate to say this because I always find someone in the audience who really likes one of the skills that I say is permanently removed. I used to really go to the library card catalog as the example of that, because I was the one that no one's going to argue that we should still do it that way. And then I found somebody who. But there are some skills that once we develop technology that can do it, we generally see us stop teaching that skill altogether. Navigating a library via card catalog as opposed to a search engine is one of those. I remember having a lesson in the library. We learned about the Dewey Decimal System and how to organize books on shelves. I don't think we teach that anymore. And my daughter hasn't learned it yet. I know I learned it before third grade. She's in third grade. So I don't think that's something that they teach anymore because it was a means to an end. It was a process, a skill that was meant to accomplish some task. And the technology that came along accomplishes the task better than the previous kind of workflow. And so we teach the new technology as opposed to that. I should probably use the logarithm tables example more often. I don't know who remembers this. Before ti83s were really, really common. I say ti83. This is the catch. All the Kleenex of calculators, but before graphic calculators were really common or just regular calculators that can do logarithms. There was a table in the back of math textbooks that would give you a whole table of different logarithmic estimations. Because, you know, calculating by hand was not something anyone could do by, you know, any middle school student or high school student could do by hand. And an estimate was good enough to learn the process kind of thing. But then calculators come along and calculate that for us. So we kind of get rid of some of these old skills that were means to an end. And I think we have to be very careful about what we Lose because very often it's the case that the person I mentioned to argued very emphatically that we should still have students use a card catalog to navigate the library. Was talking about the way in which knowledge is organized in the brain and how learning the card catalog echoed away. You're sorting knowledge in your mind. So we have to be careful about not losing some of the ancillary benefits, some skills that we teach. But generally speaking, I don't think many of us would keep the old technique in light of this new technology. The more kind of the area that I find that drives in this kind of panic area, because most of the things that get replaced by technology are usually not skills that a lot of people have a lot of strong agents to where a lot of assignments are built around. But the place where I think we see more of the kind of the force adjustment is when we recontextualize certain old skills. And so I like to use graphic parabolas as the example of that, to abuse the calculator analogy. But when, you know, we still teach students how to graph parabolas by hand, even though calculators can now do it just as well, just as fast. And we do that for reasons. And I think one thing technology forces us to do is really drill down on what we want students to get out of that process and kind of understand. When we want students to graph parabolas by hand, what we're really focusing on is a connection between what a formula represents and what it represents in the real world and what it represents in the graph. To understand how to use these formulas, how to use graphs in a problem solving process, and to have a kind of deep understanding of the mapping between two. Because all you learn is how to plug in the numbers into a calculator and get a picture out. Then you don't really understand what it can be used for and can't really diagnose when there's an error in it. So, you know, if you're graphing something and you understand for this kind of problem, what I expect to see is a peak somewhere along here where that's going to maximize. Thinking back to the classic problem from algebra, that's going to maximize the size of my yard kind of thing, and you plug it in with the wrong sign. If you have a deeper understanding of the mapping between the two, you can diagnose and say, okay, I know that parabola is supposed to be shaped like this because it's representing this kind of problem instead of shape like this. I must have put something in wrong, it's probably something like this. I can go back and debug fix. Whereas if all you know is the surface level knowledge of type, you know the formula in, you're kind of lost when you. You kind of know something's wrong, but you don't really know what it is, which is actually. So one of the classes I teach is instruction computing. And this is exactly what I see from students who are over relying on ChatGPT is that it's great as a way of getting started on problems. Getting a little bit of debugging help, getting over that initial first Humphrey. But the students who are over relying on it, when it gives them a perfect answer the first time, they're fine. But if it has even the slightest little error in it, they're helpless as far as debugging it. They can just tell it, hey, that didn't work. But that doesn't always actually get them any closer to the answer. In fact, my experience has been it gets more and more complicated and not any closer to being effective. And so that's, you know, if they don't have that deeper knowledge of how to work with the tool, they don't develop that. They don't have to develop that understanding and can't actually accomplish what you want them to accomplish. The exciting place is when we see entirely new skills unlocked. And this is where I think we go with AI. It's kind of dovetails with the next kind of area. But once we have access to some of these new things, where we go over time is we expect students to achieve things they could have never achieved without this new technology. Calculus, again, are a great example of that. What we expect students to learn in like a high school calculus class today is so far beyond they learned in high school class calculus 40 years ago. And it's not because calculus has changed. I don't think it's changed that much. And it's not because students have gotten smarter. They've gotten a little bit smarter, but they haven't gotten so much smarter to cover why. You know, we expected it more, but the technology has come along far enough that in problems that used to take students even at their best, an hour to do can now be done in five minutes with some technological support. And as a result, we can have them practice on those more, do more of those, and then move on to tackling more advanced problems than they would have been able to without that. It's kind of reminiscent of the, you know, we always talk about, we use technology to improve how we interact with the world. But the technology then comes along and fundamentally changes the world. You know, cars come along and we can use them to get to our destinations faster. But it doesn't just save us time. Now we expand our range of where we're willing to go because it can take, you know, the time we're willing to invest in transportation can take us further. It doesn't actually end up saving us the time of traveling. It just expands where we're willing to go. It opens up more opportunities without actually saving necessarily the time that we thought it was going to initially. There's a How is it? Biker. There's an analysis that was fascinating about the impact of the washing machine on the time it took to wash clothes in domestic households back in the 40s and 50s, essentially finding, you know, what were people spending their time on now that they didn't have to spend so much time washing clothes and finding out people were spending the exact same time washing clothes that they were before the washing machine came out. It's just now you only wore clothes once before washing them, as opposed to wearing them four or five times before washing them. The expectations for what we should do rose with technology as opposed to the technology saving us the time that we would have anticipated. We've started to see, at least in some of our analyses, something similar as well. But I'm curious about how durable this is and how much we can really assess it, because it's something that's kind of hard to measure in time. But our early evidence in a few of my classes is that students are using AI. They're using AI heavily, and they're using AI to achieve just as much as they would have without the AI faster. And so what we've seen is that our average number, our average grades, haven't changed for many of our assessments. With the advent of AI, the students who are going to get a C are still getting Cs, but they're getting a C in less time. And so it's kind of, you know, what do you do with that? You know, my attitude is that as long as their grades still represent C level knowledge, I'm okay with it. You know, if you're. If you know that you have C level knowledge and you're happy stopping there, that's fine. As long as your grade is still accurate to your understanding, I don't really care if you get there significantly faster. Which goes back to kind of some online education research were done as well. I teach an online undergraduate class as well. And one of our main findings was that Students in the online undergraduate class were learning more, but in less time, which I considered, you know, the best of all possible outcomes. But the reason for it was they were spending far more time doing actual coding. So when you compare what they were doing in a traditional version of the class, students were spent about 40% of their time actually writing code, whereas in the online version, it was about 80% of their time was actually writing code. Based on some things we could do in the online interface. But I digress a lot, actually. But. So that's the other. That's the place we eventually get to is this idea of entire new skills, entire new levels of ability being unlocked by these new technologies, which is where I think we're starting to see AI get to nowadays. Just very early indications, I think AI, especially things like ChatGPT, are in this area where they do something that we so fundamentally consider human like, and they do something that is so flexible and generalizable that it hasn't been quite as straightforward to make these adjustments as it was with the calculator taking the same assessments and just saying, okay, this thing that used to take 30 minutes now takes five, so do it six times. It's not quite as monitorable as some of that was, but I think we're starting to see some of these really interesting things come out. Both expect students to do more, do more research, do more investigation faster, as well as to treat AI as a learning outcome in and of itself, which is the other thing we kind of. We kind of adjust. So that was all about adjusting what we teach. The how we teach kind of for AI, at least, I think gets into three general areas of how I at least like to organize this, which is using AI to teach students, teaching students using AI, and teaching students to use AI in those first two, there's a difference, I promise. It's just. I always get them mixed up. The first is this idea, and this is the one I think is really underappreciated in some of the discourse around AI in education. Most of the articles we read, most of the conversations we have, are all about how students are using AI, and some of it's about how students are using AI to cheat. Some of it is how students are using AI to cheat, quote, unquote. Because if we don't call it cheating, you know, there are questions as to what behaviors are actually cheating, which ones are not, as well as how students are using AI to get certain advantages. And lost in that is all the things that AI can do to actually help us as teachers, which I think is the far more productive area to look into because there's so many ways that AI can help teachers with some of the things that we work on. One of the examples I got from one of our teachers I talked to at one of the talks I've been giving around at some local schools was this idea of when there was a Supreme Court decision that was handed down. When was it? It was a camp, it was last year. And the teacher essentially was a civics teacher and wanted to give a talk on that topic, but didn't have background on that topic. And so essentially just went to ChatGPT and asked some questions about, you know, can you give me a little more about this? Can you give me historical context about this? And very quickly got up to speed with a decent amount of fidelity. Now, the key thing here is that this teacher was an expert in the area. And so when ChatGPT made up a couple details, she was able to actually vet it and detect whether or not certain things were accurate or inaccurate. And so there's an expert filter that went into there, it was very important, but was then able to go into the classroom and lead a discussion, lead a unit on this decision in its historical context right when it was happening. And that's one of the key things about this kind of this area. Especially as we see students using AI more and more. One of the things we know that hedges against that is, is to get students to do things that they're really excited about, they're really interested in, and don't want to rely on too much outside assistance to accomplish. And a recent report decision that affects their lives, it's going to affect their future is one of those things that will get them more engaged in some kind of thing that's from the distant history, they don't really understand the immediate kind of impact of. So some of these things that allow teachers to very rapidly see some class activities, see some discussions and run their class with the benefit of kind of a pretty smart, I should say a very hard working, but ultimately contextless teaching assistant is kind of the way that these things can serve this kind of role. Much more in AI education has been used in this area of teaching students using AI, which is the kind of AI and education for ages has gone into this to a great extent. The idea is how do we actually use AI directly in the loop with students. And my favorite area of this is I think AI has the ability to usher in kind of a golden age of informal learning that historically many of the assignments that we find Most interesting to give to students are those assignments where they have some room to explore, they have some room to guide their own interests. But that's very often kind of intimidating because the amount of work necessary to find a source to investigate what you're looking at can be pretty significant. Some of these things nowadays that you can just go in and drop in a question about something you've always been curious about and get an answer at your level of expertise and your level of knowledge and ask you to tailor back and forth and dive into details on what you yourself want to see can really allow students to take a lot more control of their learning. And the question then becomes how we then try and connect that and dovetail that with some of the assessments that we actually have to do. But that's a place where, you know, it can actually help us as teachers as well. One of my was a PhD student. I was one of our research faculty threw out the idea of if you're a teacher and you want students to do an essay on a topic that's near and dear to them, and you're worried about them over relying on ChatGPT to just generate an essay on some topic and turn it into a whole blog, what you can now do is get all their essays and you yourself go to ChatGPT and just say, hey, give me a 10 question quiz on this topic and give it back to that student about their specific topic to find out did you understand this topic beyond just what was present in your essay, or was your knowledge very superficial? Or did you not really understand this topic at all because you aren't able to complete a pretty low level assessment on this thing that you claim to have done the research on. It's kind of a plagiarism check. It's kind of a measuring knowledge, different levels kind of check. And it's something we can do relatively scalably because asking every teacher to write a different quiz for every student in the class would be far too much work to do. But with this assistance, we can really kind of pursue that personalization at an individual level while preserving some of that assurance of what students are learning and what grade they deserve. And finally, I've got two soapboxes. I'm the first soapbox is that I think we also need to really focus in on teaching students to use AI responsibly and effectively. This is one of those things that, you know, students will compare notes. If you're on TikTok, you can see all these things about. Here's how you can use ChatGPT to cheat on your college essay, things like that. Actually, one of the talks I gave that was similar to this was to an enrollment consortium that was concerned about the use of ChatGPT on admissions essays and what they should do about that. And so we know students are talking about it, we know they're going to trade notes on it, they're going to use these kind of things. It's on us to make sure they understand how these can be great tools for learning and really healthy things to engage in really good things to help them out if used correctly. Part of that is understanding how to use it correctly, and part of that is understanding all the things they may not be aware of about how it works in the hood and how that can impact their daily kind of experience. One of the examples from a course I have on AI and education goes into this with one particular example from the previous slide about exploring something in the solar system, about how it starts at the level of just asking ChatGPT for an explanation of the hexagonal clouds around Saturn's poles, which is a really fascinating thing. It's one of those examples that you can bring up in science class in middle school and students would kind of latch onto because it just looks cool. And so you ask for an explanation of how that happens, and it gives you an explanation. And the explanation is usually a little bit too advanced. And so you drill down a little bit, say, can you give me a simpler explanation? And wait, how does that work? How, you know, if it's another planet, how are they able to do that research? And so you drill down more and more and it starts to give you examples of, well, so. And so at this university did this experiment, they've done this.

And so.

And so at this university, this research lab did this and found this. And all of that is completely made up. These are real people. But it's not the work that they themselves did, it's not where they themselves published, it's not the papers they themselves had. And it's a great example for students of how over time you see a drift and as you get to more and more detail, you get away from the kind of accurate gist where these systems really kind of thrive to understand how that is. And that then leads into, I think, a good conversation about why does it think, quote unquote, that that person wrote that article when that person did not write an article by that name. In fact, an article by that name does not even exist. And that gets you into kind of this area of being able to talk about how it works, which is the start and exploration of the strengths and weaknesses and how it can be used kind of going forward. Also understanding how these things are going to intersect with so much of the media that our students consume, so much of the world they interact with, and to understand what that impact is going to be and what they can expect from the future, as important as anything else that we can teach them. So I said, that was my first soapbox. That's my first soapbox, is we should teach students more about how to use AI effectively, efficiently, safely, etc. The other soapbox I have is that there's a. There's a trend in the conversation around AI that I push back on pretty heavily. And it comes. You can see it in these. Kind of. The way I broke this down, all these skills that I talked about as being either skills that are removed or skills that are introduced are very functionally oriented skills in the sense of they are meant to allow students to accomplish something that has some value to society, that has some value to the workplace, has some value to contribute, et cetera. Like that. And a lot of the kind of the ennui, the disappointment, the depression that I see around AI is this idea of, why would I bother to create music anymore when AI can just do it better than me? Why would I bother to make art anymore when AI can just do it better than me? Or even, you know, things that kind of have. That they're online. Why would I bother coding anymore when AI is better coded than me? For people who really like to code, I like to code. Even if I know that I could just go out and get, you know, an answer from somewhere on the Internet. There's some problems that I just enjoy coding the solution myself. And there's this undercurrent I've seen in students of, well, the AI can just do it, so why should I bother? And it ignores this fourth category of skills I think is really important for us to prioritize and emphasize to our students is that there's some skills that we teach students not because of the value they bring as, you know, a workplace skill or anything like that, but because of the value they bring to them individually. Things like making art, making music, writing code, if that's something I can do. Writing stories. I've had the question of, you know, my daughter loves to write little stories. And I've had the question of, well, AI is always going to be better at it than her, so what kind of career is there going to be for her in that kind of area, she wanted to be an author. And as far as, you know, being an author, when this intersects with kind of workplace replacement and salary and everything like that, that's an extremely important conversation to have. But she doesn't want to be an author because she wants to sell a million books. She doesn't want to be an author because she doesn't want something her job. And she wants to be an author because she wants to write stories. And the fact that AI can do that too doesn't take anything away from the joy she finds from writing stories. And that's something especially as AI continues to. I think this is the place where, you know, many, many conversations around AI talk about how historically technology has created far more jobs than it has replaced. And so that's kind of been the silver bullet counter argument to AI taking jobs is that, well, historically that hadn't happened. And I personally, I think it's a little short sighted as a counterargument because historically no technology in the past has sought to do everything that humans can do, which is the stated goal of artificial intelligence. And so I don't think we can necessarily generalize from that past. But what AI can't do fundamentally is enjoy things on your behalf. And so emphasizing to students developing skills, they enjoy practicing, enjoy exercising, that bring them fulfillment is going to be a very important thing. Because if we do this correctly, then we're going to open up a lot more time for people to pursue those kind of things. Hobbies aren't going to be that thing you do at the end of the day on Saturday, after you finish your work and finish all your household chores and things like that. That's gonna be something you can pursue. And it doesn't have to be something you figure out how to monetize. It can be something you do just because you want to. And so I hope we emphasize to students that kind of component. So that is my again soapbox about AI and education. Like I said at the beginning, most of my work is on our online master science and computer science program. I mostly work in teaching at Scale. I need to update my biography, the general chair for this year's Learning at Scale conference as well. But now it's been so long that I'm now the past chair, not the vice chair. But so if you have any questions about that as well, most of our writing has been about. So the two books on the left are about how we operate learning at scale in our program and kind of a model for how to teach at Scale future. Third one is coming out in a couple months. It's a book, and the course on the far right is kind of a course version of the book, if anyone's interested in a little bit more about how AI and education intersect and how we can adapt to things. And I will take a deep breath and then take questions.