What Happens After the Homework Apocalypse?

Today on the AI Daily Brief what comes after the AI Homework Apocalypse? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show Notes. Hello friends. Happy weekend. It of course being the weekend and a long weekend at that means that we are doing an AI long read. And this week we are back with Professor Ethan Malik who who is writing about post apocalyptic education. This seems extremely relevant to me given that it is Labor Day weekend in the US Meaning if schools aren't back in session yet, they will be by the middle of next week. So let's read this piece and then we'll come back and discuss it. Last summer I wrote about the Homework Apocalypse, the coming reality where AI could complete most traditional homework assignments, rendering them ineffective as learning tools and assessment measures. My prophecy has come true and AI can now ace most tests, yet remarkably little has changed as a result, even as AI use became nearly universal among students as of eight months ago, a representative survey in the US found that 82% of undergraduates and 72% of K12 students had used AI for school. That is extraordinarily rapid adoption. Of the students using AI, 56% used it for help with writing assignments and 45% for completing other types of schoolwork. The survey found many positive uses of AI as well, which we will return to. But for now let's focus on the question of AI assistance on homework. Students don't always see getting AI help as cheating. They are simply getting answers to some tricky problem or a challenging part of an essay. But many teachers do. To be clear, AI is not the root cause of cheating. Cheating happens because schoolwork is hard and high stakes. And schoolwork is hard and high stakes because learning is not always fun and forms of extrinsic motivation like grades are often required to get people to learn. People are exquisitely good at figuring out ways to avoid things they don't like to do, and as a major new analysis shows, most people don't like mental effort, so they delegate some of that effort to the AI. In general, I am favor of delegating tasks to AI, but education is different. The effort is the point. This is not a new problem. One of the first uses of any new technology has always been to get help with homework. A study of thousands of students at Rutgers found that when they did their homework in 2008, it improved test grades for 86% of them. See, homework really does help, but homework only helped 45% of students in 2017 why the rise of the Internet? By 2017, a majority of students were copying Internet answers rather than doing the work themselves. The homework apocalypse has already happened and may even have happened before. Generative AI why are more people not seeing this as an emergency? I think it has to do with two illusions. The first illusion is the detection illusion. Teachers believe they can still easily detect AI use and therefore can prevent it from being used in schoolwork. This detection illusion leads educators to rely on outdated assessment methods, believing they can easily spot AI generated work when in reality the technology has far surpassed our ability to consistently identify it. No specialized AI detectors can detect AI writing with high accuracy and without the risk of false positives, especially after multiple rounds of prompting. Even watermarks won't help much. People can't detect AI writing well. Editors at top linguistics journals couldn't. Teachers couldn't, though they thought they could. The illusion again While simple AI writing might be detectable, there are plenty of ways to disguise AI writing styles through simple prompting. In fact, well prompted AI writing is judged more human than human writing by readers. You can't ask an AI to detect AI writing even though people keep trying. When asked if something written by a human was written by AI, GPT4 gets it wrong 95% of the time. There are still options to preserve old assignments teachers can return to in class writing, asking students to demonstrate their skills in person or other techniques that might mitigate AI cheating through close monitoring. But for the vast majority of teachers, doing so requires adjustment and changes that have yet to be made. To date, few have actually reacted to the shattering of the illusion of AI detection by shifting how they approach teaching and assessment. While teachers grapple with the detection illusion, students face their own illusory knowledge. They don't actually realize that getting help with homework is undermining their learning. After all, they are getting advice and answers from the AI that help them solve problems, which feels like fluency. As the authors of the study at Rutgers wrote, there is no reason to believe that the students are aware that their homework strategy lowers their exam score. They make the common sense inference that any study strategy that raises their homework quiz score raises their exam score as well. The same thing appears to be happening with AI, as a study by some of my colleagues at Penn discovered. They conducted an experiment at a high school in Turkey where some students were given access to GPT4 to help with homework, either through the standard ChatGPT interface or using ChatGPT with a tutor prompt. Student homework scores shot up, but the use of unprompted standard chatgpt to help with homework undermined learning by acting like a crutch. Even though students thought they learned a lot from using ChatGPT, they actually learned less, scoring 17% worse on their final exam. Despite this, the survey I quoted earlier found that 59% of teachers see AI as positive for learning, and I don't think they are wrong. While just using AI as a crutch can hurt learning, more careful use of AI is different. We can see signs of this in the Turkey study, which found that giving students a GPT with a basic tutor prompt for ChatGPT instead of having them use ChatGPT on their own boosted homework scores without lowering final exam grades. Plus, a study done in a massive programming class at Stanford found that use of ChatGPT led to increased, not decreased exam grades. And of course, students are not using AI just to do their homework. They are getting aid in understanding complex topics, brainstorming ideas, refreshing their knowledge, creating new forms of creative work, getting feedback, getting advice, and so much more. Focusing on just the question of homework and the illusion it fosters can discourage us from making progress Today's episode is brought to you by Venice. Venice is a private, uncensored, generative AI app. It accesses open source models to enable text, image and code generation without the fear of being spied on or having your data exploited. Discuss anything with Venice without concern about it being monitored, sold, or given to advertisers and governments. Venice is different because your conversations and creations are kept securely within the browser, never stored or accessible by Venice. Unlike other AI apps, Venice won't tell you what's okay to say or not. Venice won't patronize you. It simply provides direct access to machine intelligence. No topics are off limits, no ideas are taboo. With Venice, you're in control of the AI as you should be. Pro subscriptions are available for $49 a year or $8 per month. AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit Venice AI NLW and enter the discount code NLW Daily Brief. That's NLW Daily Brief, all one word. Today's episode is brought to you by Superintelligent, which is of course our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever. When you sign up for Super Intelligent between now and the end of August using Code so back your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched super for Teams. So if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using superintelligent as an individual, once again, if you sign up for superintelligent between now and the end of the month using Code Soback, you will get your first month 100% free. Go to be super AI and check it out today Section Encouraging Not Replacing, Thinking to do so, we need to send our teachers on the process of using AI rather than just leaving AI to students or to those who dream of replacing teachers entirely. We know that almost three quarters of teachers are already using AI for work, but we have just started to learn the most effective ways for teachers to use AI. A recent deep qualitative study of teachers found that teachers who used AI for both output create a worksheet, develop a quiz, and to help with input, help me think through what makes a great American novel, Give me ways to explain positive and negative numbers get more value than if they use AI for producing output alone. This points to a useful path forward in AI for education, using it as a co intelligence and tool for helping humans do better thinking. Increasingly, AI is being used in the same way for students, pushing them to think rather than use AI as a crutch. For example, we have released multiple prompts that instructors can customize or modify for their classroom. These sorts of prompts are designed to expose illusory knowledge, forcing students to confront what they know and don't know. Many other educators are designing similar exercises. In doing so, we can take advantage of what makes AI so promising for teachers its ability to produce customized learning experiences that meet students where they are and which are broadly accessible in ways that past forms of educational technology never were. The integration of AI in education is not a future possibility it's our present reality. This shift demands more than passive acceptance or futile resistance. It requires a fundamental reimagining of how we teach, learn, and assess knowledge. As AI becomes an integral part of the educational landscape, our focus must evolve. The goal isn't to outsmart AI or pretend it doesn't exist, but to harness its potential to enhance education while mitigating the downside. The question now is not whether AI will change education, but how we will shape that change to create a more effective, equitable, and engaging learning environment for all. All right, thank you once again to Ethan for another great piece. I think that Professor Malik here is exactly right that the key transformation in the short term has to be what our expectations are around how education works. The good news is that education has been so desperately needing an update for so long that that while painful, it's hard for me to imagine that if we do this right, things don't get better. The challenge, of course, is that it's not just students who don't like mental effort. Teachers who are already overworked and underpaid as a general rule don't in many cases have the time and space and in other cases the appetite to try to do things fundamentally differently and fundamentally different is what AI is going to require. These are real challenges. How do you teach what good writing is when ChatGPT can just write for you? My instinct is that the only answer to questions like that is to go much deeper on the question of what good writing is, to get deeper on things like narrative. But creating those new frameworks isn't easy, and it's going to take time. And in the middle of that, there's no way this isn't just weird and challenging in the short term. But look, for creative educators, this will represent a new opportunity unlike anything we've ever seen. And it's hard for me not to be optimistic in the long term. Let me know what you think in the comments, either on Spotify or on YouTube and I and until next time, peace.