The Challenges AI Poses for Learning How to Write

Hi there. I'm Diane, and what you're about to hear is a conversation Michael and I recorded with our guest Jane Rosenzweig as part of our series exploring the potential impact of AI in education. This is where we're interviewing optimists and skeptics, and I loved spending time with Jane, who's a true expert in teaching writing. I keep thinking about a few of the key ideas from our conversation. One of them is, why do students even need to Write anymore? Arguably, ChatGPT, Genesis, Claude, and the others are literally designed to write and likely a lot better than most people. So what's the purpose of writing?

Specifically?

Teaching students to write. No wonder I'm still thinking about it, because as a former English teacher, this feels like a big existential question. The other one that's sticking with me is we talked about this idea of AI optimism, arguing. Arguing that, you know, this is. This is a chicken in every pot. And honestly, who knew we'd be calling back to Herbert Hoover's campaign slogans? But here we are at this very strange and interesting moment in time. And honestly, I can't wait to unpack all this that we're learning and talking about with Michael. But until then, I think you'll really enjoy this conversation that we had with Jing. Hey, Michael.

Hey, Diane. It is good to see you. And I've been reflecting between the conversations on how each episode that we've had in this series on AI has been so different. I'm just marveling, frankly, at all the different perspectives and viewpoints and levels at which people tackle our questions around AI and education in ways that I had not anticipated at all. And I'm also pretty certain today will be no different, which excites me.

Michael. I totally agree. And, you know, when we first conceived of the series, we called it a miniseries, but as we've gotten into it, we keep thinking of more people who we want to interview because we are hearing such a fantastic range of perspectives. And. And I will admit, when you proposed our guests for today, I got super excited because my favorite thing to teach when I was teaching was writing. And it was my favorite because students made such meaningful and tangible progress. It was super rewarding as a teacher to be able to give feedback and, like, literally watch them grow, you know, in a matter of days. And that's why I'm so excited to. To welcome Jane Rosenzweig to the show. Jane is the director of the Harvard College Writing center and a longtime expository writing instructor, and she's been writing a lot about the impact of AI on WR in particular, and what we may lose within this age of AI. And since 2023, she's also taught a course called to what problem is ChatGPT the solution? Super excited to dive into this conversation. Welcome, Jing.

Thank you. It's great to be here.

Yeah. Well, we really appreciate you saying yes to the invite. And before we get into a series of questions around AI and education specifically, we'd love you to actually just start off sharing with the audience and with us. How did you decide or get pulled into this topic of AI so deeply itself? Like, what sparked you down this journey where you're now contributing to the Boston Globe? It seems constantly with, like, really interesting perspectives about how to think about these questions.

Yeah. So like many good things that happen to all of us who teach, this began for me with a student. It was actually one of the writing center tutors. So about, I would say about a year and a half before ChatGPT was released, I had known that there were sort of ways, things in progress that were going to try to automate writing. And every now and then someone would call me and say, you know, do you want to work for this company that's going to automate writing? And I would say, probably not. But I hadn't really been diving into this. And one day I was in my office and I was looking at something called Jasper. I believe it was just one of the earlier AIs. And one of the writing tutors was standing in my doorway chatting. And I said, hey, do you know. Do you know anything about this? He was a. He's a computer scientist. He was studying computer science. And he said, oh, not that one, but here's what you need to know about. You need to know about the GPT playground. So right then and there, he came into my office and he showed me. So what was the. The precursor to GPT3 was the GPT Playground. And it was a little bit different. You had to. It wasn't a chat interface in the same way.

It.

To kind of figure out how to prompt it. And so I started playing around with that. I started giving it my assignments, my writing assignments, just to see what it would do. I was trying to generate a paper about an article by Michael Sandel that my students were reading. And I just, I started to see, oh, yeah, this is. This is something, right? So. So I started thinking about it and this went on for a while. I was just kind of expecting, experimenting. I didn't know that ChatGPT was on its way. About a week before ChatGPT was released, I published the first of my Boston Globe pieces. It was called what We Lose When Machines do the Writing. And it was all my musings of how I'd been trying to get GPT Playground to write this Michael Sandel paper, among other things. And then a week later, when ChatGPT came out, I was suddenly the person who knew more about this than a lot of my colleagues. Right. Because I spent all this time with it, and so suddenly I was, you know, I was an authority. Not in a very small way, because I knew what it could do in terms of writing. And then when I published that piece, I sent it to a friend at the Berkman Klein center, and he invited me to come over to a conference they were having. And I just started becoming part of the conversation very quickly. Yeah. And then it went from there. The rest is history, whatever they say.

Wow. Wow. Well, so let's zoom out then for a moment and before we get into the topic of, you know, these op EDS that you wrote for the Boston Globe, specifically, what do we lose when machines do do the writing as you just reference. And I didn't realize, I guess, mentally, that it appeared literally the week before Chat GPT came out. That's unbelievable. But I would love you to make the positive argument for AI in education, even if it's not your personal point of view. Right. Sort of. What's the best case that you've heard around where AI can enable us to do things for students that maybe we wouldn't otherwise be able to do? Or what can it positively impact, even if you don't necessarily buy into that viewpoint?

Sure. So I should say my expertise is teaching writing. And these arguments about what AI can do in education certainly go way beyond what I've spent my career focusing on. So I think that's important to note. I also think I've heard a number of these arguments, and they seem to be changing depending on what, you know, what the market seems to be interested in to a certain extent. So I'll just. I'll talk about what. What I talked about with my students in class today. Today they had watched Saul Khan's TED talk about how AI might save and not destroy education. And so we had a really interesting conversation in class. But I think of this as kind of the chicken in every pot argument. Right. So AI, the positive view of AI in education goes something like everything that every individual student needs can now be delivered by some kind of AI chatbot. So he talks about how there's a shortage of Guidance counselors. AI can be your guidance counselor. You need extra help in math. AI can be extra help in math. You need a teacher. AI can be a teacher. Oh, wait, you only need help, you know, generating some brainstorming. AI can be a brainstorming partner. So a kind of positive that it sounds like the people who are making that argument are saying, you know, the dream is. It's whatever we need it to be in whatever moment, we need it to

appear hero, as I hear you saying that. And so that makes me curious, like, you know, if that's sort of the chicken in every pot argument, what parts of that do you in fact believe? Or maybe the better question is like, are there parts of it where you're like, yeah, there's facsimiles of that that I think are right, but I would modify it in this way to make it, you know, yes, that could be a positive.

So I'm skeptical of any argument. I mean, I teach academic writing. I teach academic argument. What I'm asking of my students all the time is for evidence to support their claims. I'm skeptical of any argument that goes so big without any accompanying evidence. So, I mean, there's certainly evidence. You know, we see. There's some really interesting pilots going on at Harvard, One in the Physics Department. We've absolutely seen evidence that people think AI can be useful in small ways. But this chicken in every pot argument,

I mean, the only.

There is no evidence for this, as far as I can tell, that it's really going to solve every single problem that everybody has. And yet that's all often the way this is being presented. Not just that this is a tool that might be able to help people learn a difficult kind of physics, which I buy. Right. It's. I've looked at the. The results of their little pilot. It seems very useful.

That study was pretty interesting. Yeah.

Yeah. So that's not. That's very different from the kind of AI can be a personalized tutor argument, which seems to lean heavily on the personalized without a lot of evidence for what that actually looks like? And so I, yeah, that's where I would like to, you know, I would like to put everybody who's making these claims through the kinds of assignments I put my students through. Okay, give me this arguable claim now. Show me the evidence. Show me the counter arguments. But I don't. I don't think we're there yet. And, And I think the argument for the positive outcomes of this is way ahead of what we actually know about the technology at this point.

Heard of classic Silicon Valley. We, we tend to oversell well in advance before we. We have any of the goods to actually prove it. It generally falls short of what was sold initially. Well, let's go maybe in a direction that is closer to your expertise and that does sort of land more where you are, which is like, so let's have you take the opposite now, the skeptical take. And like, specifically, what will AI hurt? And like, how. And although I'm sure you could make a Steelman argument here as well, you've written a lot about how claims like AI is reducing friction is actually a concept counterproductive in learning, as in productive struggle is the point. And you've already started to get into this production event evidence and thinking. So, you know, tell us more from your real area of expertise, like, what is AI hurting and how is it hurting? That.

Yeah.

What's going on here?

Sure. Well, so I, I mean, when, you know, when you teach writing. I've taught writing for 25 years. The, the first thing that comes to mind about ChatGPT and the initial conversations. Right. Were all about, well, if it can just generate the paper, why would anyone want to write the paper? Or why would anyone need to write the paper? So I think one of the interesting things that's happened is that we've had a really productive conversation, not just at Harvard, across institutions, a public conversation about why we do what we do to begin with. And being able to articulate why I would want my students to write a paper even if a chatbot could just generate the paper has been, say, challenge number one. I mean, that was always a challenge. Trying to help students. Students see why there's value in this thing that we're doing. So, so that's been a challenge. I think one of the ways that I've put it that I always talk to my students about is that I'm not asking them to write a paper because I need a paper. Right. This is the, the product versus process argument. Right. We got plenty of papers. We don't need any more. So the idea that you would go out and, and generate your paper with, with A.I. sure. Then I'd get a paper that you wouldn't have an exc. Experience. And so there's something to the, the, the writing a paper in the way that I think I conceive of it is an experience in figuring out what you think about something. That's what I find valuable. So AI challenges that just by its existence. Right. You know, needing to understand why you would bother doing this thing. And then, then we have the questions about, well, could AI help with this process in different ways? And I've had many conversations about this with a lot of people. And one of the things that people bring up a lot is, well, sure, I could just help students with the brainstorming and then they could write the paper themselves. But to me, if I say that this is all an act of thinking, this is something that I actually want you to figure out what you think. It's hard for me to see the role of brainstorming with a chatbot in those moments when maybe the productive friction should be existing, where you should be trying to. Trying to figure out between you and the text or you and the video of Salcon or whatever it is, what you actually think. So I have some concerns there as well. I think there's, you know, we could, I could just go through every argument. There's, well, could. Could AI make an outline for you? Same questions arise. Right. I don't think an outline, you know, if, if once an outline's written, then you're just, you're. You're exercising someone else's vision. I doesn't really work for me.

I'd love to dig in on this one a little because I've been working with some young people who, by the time they've come to me, they've already tried to use Chat GPT to help brainstorm with them and help outline. And I have some thoughts and opinions about what I'm seeing. That is not, not only are they not doing the thinking, but the work that's being produced is not very good. And it, it's sort of obvious to me that it's very. And it's very chatgpt. And on the flip side, we've been talking to people and I would call them sort of very sophisticated kind of experts in their field and their areas of expertise. They're having a lot of success using GPT. And so I'm wondering, is there something in between there where younger people who are kind of learning, who don't have experts, this is not as effective, and maybe it's more effective. For example, you as an expert in writing, it could be a very different tool. I wonder if you've noticed anything or picking up anything in that space.

Yeah, so I think you've really hit on something that I've actually thought a lot about. I think that when we say AI is a tool, like people say, well, it's just a tool and you could use it to enhance your writing. Well, generally when you use a tool, know what you're doing, and so you know what you need the tool for. Right. If I have a nail, I. I know I need my hammer because I know that the nail has to go into the wall. One of the things that I'm worried about with AI is that you're handing a tool, do the thing. So that's why I. I mean, if you don't know what you're trying to create, then how would you know how to use the tool? Now, I can already hear all the potential counterarguments to that as well, but I think there's something. Something really solid there. Right. Like, I'm really worried about, in a sense, de skilling my students. I want them to know how to do the thing so that if they want to bring in the tool later, they can bring it in in a way that actually works for them. You know, that they know what they're doing. Whereas if you, if you hand this tool to someone who doesn't know what a solid argument looks like, doesn't know what, what it means to connect with a particular kind of audience, the idea that they would say to ChatGPT, how do I connect with my writing instructor? You know, how. What is she looking for? And it's going to, you know, sort of draw on this predictive caricature of a writing instructor. They're not going to learn what I want them to be learning. Whereas, yeah, sure, I can use it. I don't like using it. I don't find it particularly helpful for my own work. But I could write a student paper with it that's much better than what I've seen my students write with it at this point. Interesting.

Yeah, that's super fascinating. I'm curious about taking this. One of the things we like to do is just imagine, you know, if we could all wave our magic wand and design the schools that we want, what good parts of AI would we incorporate in that school design right now? Like, is there anything worth incorporating? And as you're talking, I'm thinking about, and I'm curious about, my assumption is that when you're teaching writing to your students, you have a vision of, like, how they're actually using that skill when they leave the university. And, you know, I'm making up stories in my head right now, but I'm sure the folks that you're teaching to write are going on to write, you know, extraordinary research papers, or maybe they're even becoming journalists, or, you know, maybe they're just becoming very effective at communicating their ideas and whatever role they're in. So first of all, I should check and make sure that's true and like, that's how you think about the purpose of writing. But, like, what could you possibly do with AI that would enhance that? When we've talked about the things that's going to take away is like, if they're just trying to replace the learning that they would have to have in order to be good at those things later, I don't know if that's making sense, but hopefully you can make some sense of that.

So. Well, so sure. I mean, my students are fabulous. They go on to do all kinds of interesting things. I think it's really important. There are a lot of students who are studying STEM topics who are taking my AI based, my AI focused writing class because they're interested in this topic. They're going to write. They don't always know it yet, but they're going to write grant proposals, they're going to write. They're going to be the boss of people and ask for memos and write memos and all of these things. So I do think there's something certainly instrumental in that way about preparing students for further writing. But I also like to think that when we're talking about writing, I'm really trying to focus on how do you know what you think and why you think what you think? And this is not the only way. You can certainly have conversations, figure out what you think in many ways. But when you're asking a question the way we do in academic writing, you're asking a question, and then you're trying to examine the evidence and figure out what you think the answer is. And this is also a way of being in the world that I want my students to absorb, right? It's not just so that they can

write a memo at work.

It's so that they can look at things. They can look at a video or read a book outside of my class and bring that same kind of inquisitive mind to it. So, again, those are things that I wouldn't want to see outsourced. Whereas later on, sure, they're going to make a choice to outsource memos that they're writing at work. I think it's, you know, thinking about what, what you're trying to do. I always say, my class is called to what problem is chat GPT the solution. And this has been a really helpful framing to me in so many ways, like, why are you using it if you're using it to solve a problem that it solves then maybe it makes a lot of sense. But if you're using to do something when there's a different goal there, right. I want you to have an idea. I want you to have an opinion. Does ChatGPT help with that? And, you know, I think that that's. We're less sure about that.

So, Jane, you've just actually clarified a few things in my head for me personally on my own writing process. One of them being outlines never mean anything to me. And I think the reason why is I don't know what I think until I've written my way through the problem. So. So this has been on the couch for me.

But.

But the second thing I guess I'm curious is you've essentially noted, right, that part of the reason for writing is to help people develop this muscle of how to clarify their thinking about, you know, whatever question is in front of them, right? Whatever they're trying to figure out and what. What strikes me, right? It's not about the performance. It's about the. Or the end product. It's about the. The process. So I. I guess this. The other thing, though, that I become curious about, and I think you've written this in, in some of your writing, is a lot of. At least at the K12 level, a lot of the schools there are not making that purpose of writing either clear to students or themselves. Maybe grading around, you know, sort of judging around that purpose, right? Around the importance of the process and figuring out what you think, think as you wrestle with something through your writing. I'm curious, are there places that are getting this right, in your view? Like, do you. Do you know, K12 schools that are doing this right? And if not, how do we start to move to that world?

So I. Yeah, I think that ChatGPT, the release of ChatGPT actually created a really useful moment for us to be thinking about what we're doing when we teach writing, when we assign writing to begin with. There's always been a little bit of a disconnect that I've noticed between what my students were doing in high school and what I was asking them to do when they got to my classroom. That's normal, right? We have a transition from high school to college writing. But I think one of the things that. That's a real challenge is my students will tell me a lot that they learn to write in preparation for standardized tests, right? So there's a particular kind of writing where you are not writing to discover something. You are writing to demonstrate that you know how to do this thing. Which we sometimes call a five paragraph essay. You know, how to sort of approximate a way of, of interacting, of communicating, even if you're not actually being told that you need to say something that matters to you or that's of interest. So some of this, I mean, I understand this, I, you know, couldn't begin to suggest the what should be happening in K through 12 in terms of how we could move away from the standardized test model. But I do think that it's difficult for students to see this thing that they've always done quite well, according to a kind of rubric of do you have a thesis statement? And do you have three points in separate paragraphs? And then someone like me comes in and says, wait, do you actually believe that? What do you actually think? What about counterarguments? So counterargument is often the new piece that we introduce. Right. You can't make an argument that is going to hold up if you can't understand who might disagree with you. So interestingly, this is a place where ChatGPT was somewhat useful. In my class, I built a counter argument chatbot for my students. It was just a little pilot I teach a class about. I, I thought this would be entertaining. It forced them to go through a series of steps to answer a series of questions about their, their thesis for, for a paper they were writing and then to it. It wouldn't tell them though. It wouldn't give them any answers. It was just asking them the questions. And so they actually found it kind of frustrating. And this I think is an interesting point about how we think about AI. They, they didn't all find it frustrating, but the ones who said they found it frustrating said they were expecting an efficiency tool, you know. Right. They are used to thinking that AI, that ChatGPT is going to save themselves steps. But I had spent ages trying to make this chatbot behave more like I would behave, which was, you know, to just keep asking them and keep saying is, you know, is, you know, what about this, what about this? And then I told it to give them some potential counterarguments, but they didn't have to be correct. Right. Because I wanted them to have to engage. And so I do think there are moments where something like that might be helpful. But I think that it's really, it's kind of doing battle with the perception that AI is supposed to save you time. And what I want is for certain things to take as much time as they need to take. And I do, I think that that kind of chat bot my students are Sort of one of them said he'd rather just talk to me about it if it's going to take him half an hour anyway. But there may be some interesting scalable ways to do something like that. But this brings me to sort of where I always end up when I think, oh, but maybe there was some interesting things to do are my concerns. I think those of us who are in the classroom are very aware that there's a big difference between the way an AI tool could be used and the way it's likely to be used. And I think if we don't admit that and kind of grapple with it, then we're kidding ourselves.

Right.

Students, you know, students need to see the value in what they're doing in order to want to do it. That's the great thing that's come of this conversation about ChatGPT. A lot more of us trying to articulate what the value is of what we're asking students to do. But they have a lot of competing demands. And so in a moment, you know, are you going to spend half an hour? Are you going to ask the bot. I think this is. We just need to be realistic about this.

So cool. Such. I think we. We should end it there because such an important point. And I'm loving just the reflections I'm having already. So thank you for prompting those. We have this fun tradition, Jane, where we end each episode by sharing one thing that we're reading or watching or listening to. We try to make it outside of our day jobs if possible. And so we'd love to invite you to share something to add to our list, recommended or not.

Okay, so I have been watching a TV show called Palm Royale on Apple tv. I don't know if you know about this. It's. Kristen wiig plays a 1960, 69 wannabe socialite in Palm Beach, Florida. And I think why, you know, it's not. It's not high art, but why I've been really enjoying it is because it's not taking place now. It's taking place in 1969.

And Satanism is good.

That is awesome. I think we all need to take a vacation to a different time now and then.

Well, well, speaking of different times, I think this one might surprise Michael, given what this is not a thing I. I'm normally reading. I'm not a big canon person, but believe it or not, I'm re. I'm actually listening to the Odyssey and I mentioned on some of our other episodes that we're. We're headed to Greece in a couple of months. And so I'm diving into non fiction and fiction, you know, related to this trip to Greece. It's been a long time since I have visited this poem and this time around I'm listening to a translation by Emily Wilson, a narrated by Danes. And it's funny, I wish I would have talked to you about this Jane, because you might have some thoughts about this. But I think for well, the translation who's translating maybe such a huge difference obviously if you know anything about translation. And I am in this moment in time where I'm feeling like we need actually more female energy, at least in Silicon Valley, in my corner of the world. And so I am loving the extensive explanations about the choices that the, the translator has made and how they contrast with so many of the historical translations.

I'm.

It's just, it's fascinating and beautiful and so I'm surprising myself and really enjoying

it it I love that you picked that. As someone who took so many classics my first year and my my sounds like you're listening to my college classmate because Claire Danes and I were the same class at Yale, so there's that. And Ryan Holiday, who I just heard speak was talking about why the Odyssey still resonates centuries later, even with his six or seven year old kid just the other day, something I was listening to which so that's super fun. I confess I am just finished a book that falls back into our work lives. So I apologize. It's a book published by the Harvard Education Press titled who Needs College Anymore? Imagining a Future Where Degrees Won't Matter by Kathleen Dulaski of the Education Design Lab. I will say it provoked a lot of different thoughts for me, mostly informed I think by my growing view that we need to think a lot about how we give young people the opportunity to have real work experiences when they are students and see value of what they're doing. And it actually connects in an interesting way to what you said, Jane, around making sure that the purpose of things as well is in the foreground as opposed to in the background for learners. And that's how I'm connecting with the book first and foremost. So I will leave it at that for the moment. I suspect Diane, we will have deeper conversations on that at some point, but for now I'll just say a huge thank you to Jane. This has been a fantastic conversation, has opened my perspective on a number of things and of course thank you to all of you, our listeners. We will see you next time on Class Disrupted.

Sa.