How higher education is evolving for the AI age

All right. Hi everyone. Thank you so much for joining and supporting our virtual event series, Humans in the AI Loop, where we feature MicroOne experts to share insights on training frontier AI models. So in today's discussion, we're going to explore the role of educators in shaping the future of AI. As always, this conversation is led by our amazing chief economist and Harvard and Northeastern professor, Mark Esposito. And we're so grateful to have MicroOne expert Sasha Tomek, Associate Dean at Boston College and Starling Hunter Professor at School of Business joining us in the conversation today. So Mark will try to answer as many questions as possible from the audience throughout the session. So feel free to use the Q&A chatbot in the bottom right-hand corner. Additionally, if you'd like to enable transcription, feel free to use the captions icon in the bottom right corner. So let's get started. I'll pass it off to you, Mark.

Thank you so much. Well, welcome back to all of those attendees that have been following our series, and if you're joining for the first time, welcome. We ran already two series with another group of experts before, and this is particularly close to me. Sasha and Sarlene, they are working with me at MicroOne since a good number of weeks, months. And what we have in common is that the three of us are educators working in different kind of programs. And so this will be particularly important as we bring together people that that sometimes don't talk about AI the way we do talk about AI. And so it's kind of unique to use our collegiality to explore this. As I am asking Starlene and Sasha to give a bit of intro about your journey and how AI became part of your life, I'd love for you, Starlene, maybe to start and give us a sense about how did you start. You are an organizational theorist by training.

Yeah.

You've been a professor. You are a professor now. So beside the fact of being a professor, where was your encounter with artificial intelligence in the context of somebody doing your kind of work, your kind of research, your kind of training?

A couple of places. First came, I think, as the, as you know, late 2022, early 2023 when these models started to become commercially available and were starting to get a lot of bandwidth and people's discussion. So right around that time, I was preparing a packet for the executive education group at London Business School. And I started working that, I started using that tool as much as I could in putting together my packet. Obviously not to write things for it, but they had a very detailed template about what they wanted. But as I put together run sheets and exercises that wanted people to do, things they could do pre-program, during the program, post-program, I began to leverage Chat and other LLMs to be able to see what kind of insight they could give me and even structure my thinking or to go out and grab information. So that was really a first. And then more recently in teaching undergraduates earlier this year, it was a matter of, you know, the faculty said, oh, you know, watch out, they'll use it. So I said, okay, well, I'm going to have them use it, but we'll do it in a very structured way. They will give me their prompts and part of their grading will be also helping them understand how to structure prompts and get better answers from them. So it's been very practical for me. It wasn't part of my research agenda.

Yeah, sorry. I appreciate your journey. I think it has been common to also my specific journey. And I think as an educator, we started to see students pioneering the use of this technology because it was kind of fashionable at first, and then why should we look at them as they are the incumbents of the technology? We equally can co-share the responsibility. And I think it is great that educators do not hide under the evidence of a technology that is now rising, but they're fundamentally kind of contending, right, a space and a voice. And I really appreciate that. Sasha, I think you and I equally share, like I do share with Starling, a lot of commonalities. But similar to your background, you got your training in economics. Economics. You now are an associate dean at Boston College. Your program is also about economics and analytics. And so you could argue that AI was in your sphere of interest as part of kind of your, your research and training. But the kind of AI we're dealing with right now, I think it doesn't necessarily mean that being a professor, being an economist, you are a trainer or an expert in machine learning, right? Or in this case, large language models. So how did you get close to the technology that we are, you know, kind of helping to train every single day in our function at MicroOne.

So, you know, uh, in, in my approach, in my role, I do two things. One is I manage the Master's in Applied Economics and Applied Analytics at Boston College, uh, which we started in 2005. And, you know, I built it from 5 students to about 300 students now. So in that role, I constantly have to worry about what value are we delivering to our students, right? Because, you know, I I'm an economist. We got to deliver value, consumer surplus, producer surplus, right? So in doing that, I'm constantly on the lookout what are the skills that my students need to get the job, because this is a terminal master's degree. We just recently started preparing people for PhD, so our main focus is to get them employed. So to do that, you know, it was let's move away from Stata to R, let's, you know, kind of move away from R to Python, and you know, and better yet, let's not commit to any software in particular. We need to teach you how to think about things and then software will come along. And then all of a sudden AI burst onto the scene. And I will just give you a very short snippet. My introduction to AI was before it became fully, fully open. Somebody put the blog post about World War II in the Balkans and said there was this Chetnik guerrillas that together with Germans were fighting Germans. And I thought the guy was being particularly funny because that's how you would summarize a lot of discourse. But other people said, are you drinking? And he said, No, I'm using this new tool. And it was ChatGPT, you know, at the time. And they asked, well, is this a Google Translate thing? He says, no, I was prompting it in Serbian. And this was, I think, like November maybe of 2021, '22. And then in February it just blew up, right? So I saw something interesting there and it got on my radar. So in preparing of the students, we can't ignore it. I mean, they have to know it just like we had to teach some coding to them. If they didn't know it, they also— we now teach GenAI course that's, you know, The other part is my job as the associate dean, where I have to do market scanning. We have to figure out what new programs to put on. And that's where I see a lot of potential for basically for AI to do what it's promising to do, to cut down on time for research and such. And then finally, as an economist and as a researcher, I find it incredibly helpful in, say, give me the first run at the model, at the theoretical model. It does not come as naturally to me as the quantitative— I mean, as empirical modeling comes. So So if I have an idea and say, well, this is how I think these things are evolving, can you please give me a model? I can check that model easier than I can come up with it in doing it on my own using calculus. And then as a teacher, I think I do worry about the bad side of it, the cheating and all of that. And we can talk more about that because it scales and democratizes a lot of things. So it scales and democratizes cheating, but so it does the tutoring. Right. So it gives people resource that they could not maybe afford on their own. Right. I'll say one more thing and then throw it back to you. But just a very simple example, if my parents went to college in the United States, for example, and have graduate degrees and such, they can proofread my papers in the middle school, high school, college, and they can help me along. But if they have not, you know, it's not easy for me to get the copy editing support. Right. I mean, right. So ChatGPT just kind of democratizes that. Again, The flip side of it is it also makes it easier to cheat, but democratizes that as well. Because if I could afford to pay somebody to write my papers before, I don't need to do it now. So, you know, bottom line, it's not really introducing anything new, but it's really scaling things really, really, really fast.

Fascinating. So I like to get on the side of scale and kind of take it, gentlemen, in this direction. One of the things that I think we have all noticed how somehow the diffusion of knowledge became much more accessible and cheap. And, you know, professor used to be the gatekeeper, that level of knowledge. And we used to profess in many ways. And I think we have seen that what makes us a good professor is more about this orchestration and curation of the learning journey rather than these— so like this distribution of knowledge one-sided. But many of our colleagues struggle with this, right? Because it feels like they got debunked a bit from the entitlement that came from having a job of diffusion of knowledge. How do you think our role should be considering that this technology to some degree has given access to a lot of conversation that could be considered basic knowledge? And many of the material that many universities have been using, they naturally get outdated by the fact that they kind of capture a world that is changing at a very rapid pace. How do you see this role of professor changing through the experience of pioneers like yourself that have been able to integrate it without feeling that it was an actual trade-off. It was actually an expansion of our capabilities. So, Stanley, love to have your thoughts on this.

Yeah, well, you know, one of the things that I have the fortunate position of being able to teach topics like organizational network analysis, which a lot of people understand intuitively but have never heard the name. Right. So, and in the same way, when I've taught at other places, maybe where I have people with lots of industry experience, but not having what we might call fully developed models of, say, how an industry works or a theory of a firm, right. So I can start with people who are quite knowledgeable. And this thing about guiding them on a learning journey to say, you know, we have a conceptual vocabulary for these things. And it used to be that I would take them through the individual theories and and say, this is how we frame or conceptualize or model a firm or an industry and so on. Well, now we can do that, I think, in a way that they don't have to rely as much on me to do that. I'm still going to put the course together. And like you said, to orchestrate and think about how to— in the pacing and the structure and the things that I want them to learn. But there are many places where they can diverge off on their own that they wouldn't have done before. They pretty much would have said, you know, week 1, we do Porter's Five Forces, and next week we do something else, and we go through. And only if they were exceedingly ambitious would they go and find other theories that they could look at and related ideas. Now that will be there for them, even if they're not asking for it, because it shows up in the bottom. If I ask them to prompt or to go along a certain direction or do exercises applying some framework, they are also prompted by the LLM or given alternate paths that they can follow so they can individualize the experience in a collective journey. It's almost like everybody's taking their own snapshots, but everybody went on the same vacation. And, you know, they'll all experience the same things, but it can be a bit more personalized, interestingly.

At least it's the way I see it. Very interesting. Sasha, look, I think same question for you because I love to have your perspective on this as both as a professor, but also as administrator in a university, right? So how do you see the role of professor evolving with this technology? Because I would imagine that as there are enthusiasts like us, there are also professors that are looking at this technology with skepticism, right? So just share your thoughts on this.

Well, that's putting it mildly. I mean, there are professors that are putting it, you know, that are looking at it with outright hostility. But again, I kind of hate but don't hate to be the guy that says, hey, nothing new here. It's just scaling really fast. So if you think about from the moment we started writing things down, we've been replacing oracles, right? The people who would pass knowledge through some kind of oral way, right? So Gutenberg sped it up, you know, internet sped it up. And some of the colleagues that have been around longer than I have, they are saying they are very much reminded of the late '80s, early '90s conversations when Google came along and so on. So I'll start with a couple of things. So number one, and I've been saying this ever since I started teaching and supervising faculty We are not really in the content delivery business, right? And if any of you have seen the Good Will Hunting movie, when Matt Damon, you know, completely embarrasses this Harvard PhD in history student, no offense, Mark, and basically says, look, here's the one truth. You dropped $100,000 on education that you could have gotten with $5 in late fees in the library. And that's been true in whatever. Whenever Good Will Hunting came out in 1999, it's been true. You know, before Gen AI, it's true with Gen AI. So when I think of my programs is what am I making them? Why would you come to me and not to LinkedIn Learning or Coursera or something else where you can get content? So it actually becomes about engaging people and it's always been about engaging people, especially when you are dealing with people, right? So who will come to us for training are people that want that curation and that will only get more important with with GenAI because you now can get even more exposed to a lot of things, you know, that are correct and a lot of things that are incorrect, just again, like you could with Google searches. So they will come for that curation, they will come for that engagement, you know, and they will come for that kind of, you know, experience of being regimented in some sense, because some people sometimes, like myself, lack self-discipline, you know, to learn on our own. But that's number one. Number two is What, you know, if I say you can't use GenAI, what am I setting you up for? And I think in this day and age I'm setting you up for failure because you will not be able to compete with somebody who knows how to use GenAI properly. And it's not a matter of being able to produce a report. It's actually being able to do that critically. So actually the premium I think is moving, you know, to basically knowing when GenAI is being wrong. And you are having this kind of supervisory experience almost earlier, right? Because even when we deal with people as managers, our main talent is knowing when things look fine and when they don't, recognizing what's good product versus what's not such a good product coming from somebody, and then, and then prompting them to change it, right? But with GenAI, I can do that as, you know, as a student, I can do that as anybody, but I still have to know when it's feeding, I still have I still have to have some idea when GenAI is wrong. And if you don't get to play with GenAI in a safe environment like school, when you go to play for keeps in, in, in your career, you will end up hurt, right? Because you will produce something that will embarrass you in front of, you know, your client. It will be completely wrong and you will fail. So if we don't give you that room to experiment, in, in, in the education confines, I think we are setting you up for failure. And then within the education confines, uh, and again, one more thing and now stop. I link— I often think of Gen AI as a calculator, right? So I have an 8-year-old and I have a 13-year-old, and my 8-year-old is forbidden from using calculator in her math classes because she has to learn the concepts of multiplication, division, and so on. But my 13-year-old Nobody cares if he uses calculator because he's talking about higher-order applications. So I think, you know, whenever we teach, we got to figure out what is it we want to get across, what is it we want to assess, and then what is the role of GenAI in that. So in some classes, sure, blue books, paper, you know, handwritten, no AI. You really got to struggle with this and you got to get some knowledge. But then some other classes where you are synthesizing and researching a lot of things, you've got to know how to use AI effectively because otherwise you'll be left behind.

Absolutely. You know, what comes to my mind is actually elevating for me to hear that we got clarity about this without fearing this technology. We are able to place it for what it is, is a companion, is a tool. And I tend to agree with you, Sasha, equally, equally like a calculator sometimes. And I always also try to say, look, calculators are better than me at doing calculus, but that doesn't define them intelligent per se, right? Because I think we have been using this very narrow way of looking at intelligence from an analytical perspective. To define intelligence in general. So I think it's important, I guess, that we continue to kind of bring this message about use technology for a purpose, and the purpose has to be the amplification of your own thought process. Which brings me to fundamentally, that's why education has always been a form of elevation of people. We always recommended people to go to school regardless of their career choices because it's really about building those skills. But today, I think many programs in many parts of the world, they kind of struggle to meet the expectation of the market in many ways because when they graduate, we've been noticing more and more younger graduates struggling to get integrated into the marketplace. And, you know, gentlemen, MicroOne to some degree has done a lot of progress towards AI and recruitment. And I think they're doing an applied form of AI for good in understanding that AI can really be a better way of generating inclusive recruitment. Recruitment. But I'm wondering about institutions that are kind of pushing students through programs, and when they graduate, they might be a bit on, uh, let's say, in dissonance with where the market wants them to be. How do you think we can solve this? Yesterday in our prep call, very interesting conversation really emerged from this. Uh, Sterling, what would be your take on that?

The, the last part I couldn't hear you very well.

You said, uh, before our conversation, and I lost you for a Yeah, Meli, about how you think education or institutions should really prepare students for careers, considering the challenge we have now between career options and entry into the market when they graduate.

As we were saying yesterday, there are a couple of options. One is that I can see a shortening of time to degree so that students can get out and get exposure in industry perhaps more quickly. Another possibility which came up was actually lengthening so that there's more time spent going back and forth between work and an education so that the bracket, even if it went to 5 to 6 years, it's a developmental experience which is deeply embedded almost that the students end up being the link between us and industry so that we're cooperating more closely. And one thing I would add to that is that's not going to be just in our programs. One of the things that I thought about So, you know, as I was thinking about this discussion, is that the importance of the, the, the trivium and the quadrivium, but especially the, you know, the trivium of grammar, rhetoric, and logic are going to be so much more important, I think, going forward, because we have to learn to prompt and to structure in language. And so there are, we have colleagues who have some real distinct advantages in that regard. And as they begin to drill down and think about the structure of prompts, I think we're going to get something very rich. Right now, it's a lot of us who are science, technology, engineering, mathematics, economics who are really focused on it. But as our friends who've specialized in the art of words or the arts of word have their say, I think we're in for some really good insights.

Darling, there's something you shared yesterday that I didn't know about, and I'd love for you to share it. It's not the first time that we might consider for circumstances to shorten programs. You're telling me that something happened, probably that Duke is your alma mater, right?

Yeah, Duke is my alma mater, and they have a program called a physician's assistant where during World War II they shortened that program to 2 years because they needed doctors to be out and at the battle because the whole war was only 4 years long. So they needed— so they cut the training to 2 years. And then when it was over, they went right back, switched it back to 4 again, but they kept that program around. So it became one of the leaders in, in training this new position called a physician's assistant. And I wonder if there are parallels not only in the shortening, but also whether we get new kinds of degrees where people will be, maybe these are the new entry-level positions that people are prepared to do things after a short, intense period of time and then possibly more training later.

Fascinating. Look, I learned something yesterday when you, when you shared this, and I was thinking, wow, that extraordinary circumstances, right? They drive our reasoning to a point where we are able to justify, right? And why not now if we consider that we might have definitely a problem with the younger graduates not able to enter the market and And same question, Sasha, to you with a bit of an angle on the fact that working as an associate dean, you look at the structure of a curriculum. And we try to make sure that we're meeting the taxonomy of learning. So there's a lot of structure that is also inherited as part of our legacy of understanding what should happen. How do you see this eventually in comparison to the contingency in the real world, right? Are we somehow in contemplation of a model that kind of echoes of the past? Or do you think there's still some reason why we have to have students go into a specific set of years of training? What's your, what's your, your feel on that?

So, you know, I mean, I'm personally taking classes as we speak in marketing, and one of the quotes that I really like, and I can't remember who it was from, is basically saying, look, your customer doesn't want a quarter-inch drill bit. They actually want the quarter-inch hole. That's what they are looking for. And then you sell them the tool that accomplishes that. So the question, Mark, I think, you know, I think one of the big mistakes is putting all of the education, all the higher education in one bin. We are all very different. I mean, I'm at Boston College, which is, you know, high-level private university in the United States. And the first thing that we will tell you is actually what we do, we engage in what's called formative education, where we are actually trying to develop a person rather than provide a skill set at the undergraduate level, right? Well, my program, our professional programs are more along the lines of what you are talking about. So my master's in applied economics and whatnot. Yes, we engage in formative education, which I think to Stalin's and to your point, will become even more important with Gen AI. But we also want to give you some skills. Now, what we are seeing is we are seeing almost disappearance of entry-level jobs and deficiency of people with the experience. Why? Because the tasks that entry-level jobs we're doing can now be done by AI. But somebody needs to know when AI is wrong, right? So you have to have experience. And that's when we got into— so sure, we want to shorten time to degree, and that's being done, by the way. There is right now— accreditors are looking at the 3-year bachelor's in United States for pretty much the first time ever. But then what? I mean, I can push you through the program. If you are particularly talented, you can maybe do it in 2 years. Or if you go to certain, you know, certain schools, you can finish your high school and your college at the same time. And then what if we have not exposed you to any kind of experience in industry? I think you will struggle in the AI age. And, you know, I'm full of analogies today, but my analogy right now, I worry because we were teaching people how to do analytics and part of it is coding. But has that become the typing of early 1990s, right? In the '70s and '80s and such, you could have a career as a typist. And what it is, if you break the task of, hey, I will I want to, I don't know, provide the analysis of the company. I have to think about it. I have to go around the numbers. And then in order for people to not make mistakes because of my awful handwriting, it needs to be typed up. So it used to be when typing was complicated and expensive, right? Because you had typewriters and you could not correct mistakes infinitely and on and on and on. You would actually pay somebody for that kind of last mile, that typing. But as we developed word processors, that completely went away. We all now type. Right. We all know how to do that. Well, the coding kind of went that way too, right? I mean, now it's expected, you know, in the MBA program now we require people to code in R. I was having conversations even 4 years ago with some people who were saying we were introducing too much coding into our managerial program that we run in healthcare.

Why?

Which executive codes? Now it's expected. And with GenAI, now you have no excuse to go and ask your subordinate to take the first pass at the data because you can write, what is it I want? I want to see the trend line of my sales for the last 5 years. That used to be something you would give this kid fresh out of school to do for you. Now there is no reason to do that. So what we want to pay for is now increasing, right? And that's the story of technology. We quit paying for certain tasks as the machines take them over, whether it's picking produce or whether it's writing code. But AI is escalating all of that. So I'm getting to the point of saying we need to make sure that we can actually provide some work experiences to our students throughout the program, whether it is having classes. In our classes, I have people from industry who teach and I say give them assignments that look like work product, have them kind of learn the context so that they can be more effective using the tool, whether that tool is Python or You know, GenAI, uh, LLM model is beside the point, right? So, so we have to, to be much more purposeful about the end that we have in mind. To get them a job, is it to form them, if you will? What is it? And then work from that end, uh, kind of backward toward the beginning. Time, to the degree in, in that, you know, yeah, sure, it can be, you know, for cost reasons, obviously it can be important. And I also think one final thing, I think we are probably more rapidly. We have been moving to this all along, but I think it will accelerate. This whole idea of chunks of life where you go to school, then you work, then you stop working, go to school, it's been disappearing. And I think it's going to disappear more rapidly to where you will actually have to do things side by side. You know, you will study while you are working. So when you are done with your studies, you actually can make sense of those skills and show where you derive value in the ecosystem. You know, that's the technology and the people.

Fascinating. You know, gentlemen, we see already some questions coming, so we'll be looking at the questions soon. One question that out of curiosity, certainly I have, you, you currently do work on executive education. So you're basically teaching executives, people that don't necessarily have the problem what we're discussing about because they have a career They graduated probably a period of time where they could find a job and now they are either doing an executive education because they want to move up the ladder because of personal interest. So how do you see them positioned? What's their posture towards the use of technology? Because I would imagine that there's also generational conversation on how at work you might have different postures depending on who is there when you are talking to them, when you're teaching a class. How does the conversation on AI pops up? Because I would imagine the context of what Sasha said before, it might be, for example, inherent to a student that uses generative AI for an assignment. Therefore, we are pushed to have a conversation about this. But in the context of executives taking a class, right, how do you see this eventually just manifesting?

Yeah, a couple of ways. So as I mentioned to you guys yesterday, my Instagram feed is now filled with program announcements, offerings from all the best schools that you can think of, you know, Johns Hopkins and Penn and all up and down who are doing domain-specific cohorts and teaching them basic AI skills, prompting skills, and then how to leverage that in their own profession. And I think that needs someone who is also knows those domains well in order to do that best. So take, you know, an AI in healthcare kind of thing, or AI for people in learning and development, and then, you know, how to roll it out. So there's a need. It reminds me a little more than 30, 31 years ago when I was a graduate student taking a first course in HTML, right? Learning how, you know, all those things. And then it sort of dropped in the background where you didn't have to learn to do that anymore. So, one of the things in talking with people who have a lot of experience is wanting to stay abreast of this current development and understanding, "Well, what can it do for me?" Right? And then, there are those who figured that out but still always come up with the delegation question. So, for instance, we've got— and this is something I'm working on for one of my classes. And I use an example of a woman. You can find it online who wrote an article about how she quit Amazon. Because they were flattening, and the hierarchy— a memo leaked talked about how they were doing that, and her span of control went up to 25, and she couldn't take it anymore, right? So there are very practical things about understanding what your people are doing. I don't know if it's necessarily agentic, but helping you figure out things that you might delegate, because if you still have a span of 25, you know, you do still have people working for you. And it's one of the things that you might think about and say, how can I better allocate tasks to— and you might be able to get assistance that way. So, there are people who need help with the basic functions of the executive that should have been around for— for the people have been talking about for more than 100 years, right? That will always be the case for people that have subordinates. So, those are two areas where I'm focused in dealing with executives in this time.

Topic.

Fascinating. And so if I take the opportunity to pull into the conversation Sasha, but this time maybe guided by a question that we received from one of our attendees, there is a gentleman called Alexander Maric that has asked a question. And, you know, it happens that both of you have kids in high school right now. And so the question is, how many years would it take to have AI leaned, I guess, learn in high schools as a mandatory subject? I think the question is about training AI model builder on. And so I'd love for you to share, both of you, your experience, whether you think the high school are making— what is interesting is that Sasha, you're in Boston. Charlie, you're in Stockholm, right? So we can talk about two very different systems in many ways, but also systems that are advanced economies. So places where we could have resources to think about this in a progressive way. What is the state of integration of AI at the high school level? Do you think we're far from seeing this as a mandatory subject?

I'll let Sasha go first on that one.

I, you know, I don't see it. Sorry. I mean, I don't see it any more than I see coding as a mandatory subject. I mean, definitely high schools and colleges will develop expertise in teaching it, but I see it more as some kind of club activity, elective, whatnot. I don't see it as mandatory. I mean, when I— so again, when I came to US in '93, right, it became obvious I would need to learn to use a computer. And again, back in Montenegro, back then, typing was a skill that you would, you know, pay somebody for. And I didn't have a computer at home, so I took a class in typing, right? So it was offered. I don't know if high schools still offer it or not, but, you know, it was not mandatory. A lot of people— my point being I think this— parts of this you will have to learn on your own. Parts of this will be offered probably in a school with more resources. Like anything else, you know, the ones with more resources will do a better job of it or offer it more often. But I think the schools will come to a conclusion that it is something that by and large will be learned organically because you will have to. So I don't see it as a mandatory, but I think already in some high schools you can get help with, with any of this, if not class, then through a club or through some kind of after-school activity. But I don't see it as mandatory in college or high schools. I just don't.

Interesting. And Salim, please. Yes.

I was going to say real quick, I'm not seeing it yet either, but I've just taken it upon myself as my daughter was at the German school. So it means that she's doing courses in English and German and Swedish all at the same time. So, kids her age are doing this because they're constantly having to translate vocabularies in related subjects across languages. So, it's become the go-to for them versus Google Translate. And so, that's one of the first things. But in terms of it actually being required or used in schools and high schools, I don't have any awareness of, you know, of neighbors and friends whose kids are at that school that it's happening. At that probably are still more interested in traditional teaching of making sure that the students, as I like to say, wrestle with the material in an analog fashion first. But I hope it would come along.

I just fear, sorry, Sasha, that maybe some private institution might be able to pioneer this as mandatory. But then again, we're creating a disparity of access. That's the problem that I think we should really consider. That's why I think it was a very interesting question. I know Paula has asked a couple of questions. And Paula, I think one question was kind of answered by Sasha when we were talking about what I call resistance. And he said it was mildly put. There are definitely lots of professors who are pushing against this. But I think the question you're asking, Paula, the second one about whether it's fear that is kind of withholding the progress towards AI ethics and policy around governance, That's a great question. Look, we had a speaker last week that was, is a postdoc at Stanford who is researching governance. But I think it's great that we have this conversation with, with the business school professor, right, that we, we kind of looking at governance from a perspective also how companies are in need of redefining their oversight. So both of you, Sasha and Charlie, your sense about why do you find that sometimes we are afraid of governing this technology? Is it fear? Is it lack of understanding? Is it that it feels overly restrictive? What's your take on this?

Well, you know, Sasha, you go first if you want. It's great.

Sorry, I thought he was asking me. But, you know, I think I honestly, if I had to sum it up, Mark, I think it's just moving too fast. I mean, who has really developed expertise in AI, right? I mean, there is simply not enough time to read up on everything and to experience everything. And then, you know, new Things come too fast. And I think that's what's making a lot of us uncomfortable, you know, at every level, as professors, as professionals, as well as this kind of uncertainty. Where is it going next? What is it going to replace? And what's our role in all of that? So I think that's where it's coming. The other thing is, again, it's scaling up cheating. And, you know, in academia, I've been doing this, you know, full time for about 21 years now. And then before that, in graduate school, I was teaching and there were always colleagues that, you know, their favorite pastime seems to be catching students cheating. And this is just giving them a heck of a lot harder time, right? Because it's more difficult to prove. It's much easier to do. It's more difficult to prove. And that's, that's what they are worried about. I personally don't take that adversarial view of my students. You know, are some people cheating? Yes. Have they always been cheating? Yes. So what can we— if we take that as a fact, then what can we do about it is a different question. Throwing the baby with the bathwater. And finally, one of the things that I thought was particularly astute from one of my colleagues who is actually a theologian who works with the AI is that basically maybe we feel threatened, you know, because we are the sources of knowledge. And that's a big part of our identity as academics, right? People are looking at us to kind of provide them knowledge and answer questions and such. So then what's use of us if AI is available? But again, I would say, wasn't that the same issue with Google and Yahoo when they came along? So I think it's a very potent combination of all of that that I think is at play. But I think the speeding up and the scaling is the biggest issue because it used to be, you know, you learn STATA, you were good for 10 years, right? I mean, you could work on it, you know, but now things are changing so fast and it's very difficult to feel comfortable. Comfortable at any given point in time. And I think it's that discomfort that's driving a lot of this. Absolutely.

So I do remember a story of a professor using slides with the domestic currency even after the euro was adopted, and he was to some degree forgiven, right? Imagine, right? But this is, this is an entirely different ballgame, right? So, yes, Charlie, your take on this in terms of this resistance towards governing?

Well, there's always, you know, going to be resistance to change is just sort of a given, although there will always be people who embrace it as well. You know, when we talk about governance at the level of the models and the people who are developing it, you know, be it ChatGPT or Meta or whoever it is, you know, that's one level of governance. And then we need a second level within the institutions, and it doesn't have to be a consensus across all institutions to say these are the ways and the things that we expect. Now, of course, unfortunately, as Sacha mentioned, a lot of it's been grafted on to existing policies about cheating, but one of the things that hasn't happened maybe as much is the positive side of recognizing the opportunities that are represented for us in curriculum development and new ways to to design courses. I remember one of my faculty mentors back in the '90s thinking about what did it mean now that hypertext was available and that you could produce documents that would have hypertext and where would you take students in the way that you think about, or even in the current generation, their love for gamification of things, right? So there are opportunities to do things with your current class And I helped someone do this a few weeks ago to say, if you really want to make it engaging for kids, for young people, take what you've got in your syllabus and put it in and ask it about how to gamify this, right? And it came back with a ton of suggestions. So, those are things that they're going to be people who maybe have a little gray in their beard, who've been around and seen a couple of transitions that can say, Hey, don't fear, you know, embrace this one. This, this, I think it's going to be, I, I'm, I'm very optimistic about, about it. But, but schools are going to have to think through it well at the institutional level, departmental level about what they're prepared to accept.

Right.

That's right. That's right. It's really almost like an exercise of foresight on what we can't easily predict. Right. But, but it's naturally changing. Right. And so gentlemen, we kind of getting close to, to our time. There's a question from Resit that I will try to channel in a slightly different way. Maybe connecting to the job that we have at MicroOne in training AI. And you mentioned something, Sasha, before that I think we should definitely talk about. We do encounter a lot of logical fallacies in the models. We do find that there is a lot of imperfection in the answers. They are seemingly looking neat, but they are not carrying the substance that we'd like to have. And our expertise is we're really guided in understanding that. How do we somehow improve, um, it's like the level of literacy, not necessarily just on the use of technology, but also on the detection of these flows and these logical, uh, fallacies so that we are also creating a clear space that is a boundary on understanding that technology is not omnipotent, right? It is a technology that is an extension of our capacity to program. Um, and clearly, right, it comes with all sorts of benefit, but equally cost in many ways. So what's your, like, suggestion for people that are into the technology, but also they want to learn how to be a bit more forensic on it?

Yeah.

Mm-hmm.

I, it's tough. I mean, it's tough. I, you know, so it's just tough. And I, what I think honestly, and where I'm kind of moving with it is I will essentially clone, I mean, I guess that's what we do as academics always clone myself a little bit with my students in a sense that I will actually ask ask them, hey, here is what we are doing this week. So for the term project or something, I want you to actually give me LLM output, you know, so, you know, build a portfolio using Sharpe value, for example. I want you to put that in LLM and come back with what it provided and then go and do it right. So hopefully the day will see it is not an easy thing for the, for the model to do. The model gives you authoritatively looking, you know, whatever model you use, it always looks good. If it's properly formatted, it's very seldom glaringly wrong. You know, it's often finer points. And I think, you know, they simply have to kind of do it side by side the way we do when we do the training of these models, because we have to do the work and then see what the model provided. And then sometimes model does really well and sometimes not as well. So the students, I think giving them that experience, giving them some healthy dose of skepticism, I think is the only way to go because they have to have it in order to recognize it's not, you know, just something. And it's the same story again, you know. And then the other thing is, if you can, depending on the size of the class and such, always have some kind of discussion with the students, right? So you do the presentation. Now let's see, can you answer any questions about the presentation? But even before, generally, this was again the issue that some did. They buy the presentation online. Line from somebody else and didn't really put in any work, right? So, so I think honestly, I think that's what it is. And this is when we get to that kind of curation safe environment that we provide for experimentation education. I think that's kind of our role and we will have to just live to the realities of GenAI. But then we have to, because if they do that, then they will better understand their place as a human in that loop, right? Because I think it can get very dystopian very quickly if all you think you can do is just, you know, do mindless research and sending out, sending out reports as a person. It's not very useful, useful either. So, so that's what I think. I think giving them that experimentation space and requiring that.

It's, it's very, very deep. I think it's, it's something that, uh, you know, both Charlie and I were nodding when you were talking. It's really the fact that we humanize ourselves by using this technology if we're able to understand it for what it is, right? Rather than feeling there is an unnecessary tension against interested, on the contrary. So, Stanley, your take on that, on basically the reasoning part, right, with this technology.

Yeah, I think that there's a lot to learn. I have a colleague, we went through graduate school around the same time, you know, very well known, written lots of books, articles. You know, she's done fabulous. And she had a couple of, you know, statements or posts on LinkedIn in the recent few months about how, why didn't AI give me the answer when I asked it? This question, right? And then I saw one just the other day, which you'll see in my feed if you look, that I've commented on, but I didn't give the answer yet, which is a guy posted a geometry problem, right? And he said, oh, you know, this LLM is supposed to have PhD-level intelligence, scoff, scoff, right? And so I looked at it and it's been a while since I was in high school geometry. So it took me about 15, 20 minutes, but I broke it. I asked him to share his prompt. He wouldn't do it. He didn't I couldn't do it. But I broke it down into 3 pieces. You probably have an idea of what that looks like to find, you know, some interior angle or some acute angle, right? But I had to break it down into pieces. I had to explain it like I was explaining it to a blind person. Right? And then I got the answer. I got the answer to come out after a few tries. And so thinking about how, at a very sort of micro level of how break knowledge into pieces. Because I understand we want a holistic answer, or when we're experts, we can jump the intermediate steps and see it. And I like the idea about being able to help students especially break things down into pieces so that they can follow the logic and think, how do you go through this problem to solve it? But also being aware that understanding is a holistic thing, and that you don't want to break it into so many pieces that you're doing well, to switch metaphors, like explaining a joke, and then it's no longer— it doesn't have that impact anymore. It's no longer holistic because you broke it into so many pieces. So, you know, that's, that's, you know, sort of the way I think about this garbage in, garbage out, which was part of that question, which is that we can do a lot to learn to properly structure and state our assumptions up front. And if it's still not working, and we all have some experience with with that, then we can be very clear about what's understood and not understood.

That's fascinating. And there was a recipe there, that question. So thank you, Ressi, for doing this. Um, guys, look, I— we are at time, and it's quite amazing that almost an hour has gone by in such a pleasant conversation. I think the only thing missing was a cup of coffee or a glass of wine, depending on the time. But I really, really, first of all, appreciate I appreciate this intellectual conversation that we had. I appreciate the collegiality. It's an honor that we are colleagues of the same project at MicroOne, and also is a testament to the fact that MicroOne has been able to put together amazing profiles that usually do not converge in the same place, in the same faculty offices, right? So it's quite amazing that we are, we are able to do that. I invite all of you listening to kind of follow the work that both Sasha and Starling do. Both of you are on LinkedIn. Get in touch. I think none of us mind getting in touch and talking to people. Um, and look, I, I look forward to the continuation of the series. Um, we will continue to have people joining us. Uh, this is coming quite soon. Uh, this is on the 10th of September. Uh, so Jemmy Pack, she's the director of the Reconfigurable Robotic Labs in EPFL, which is a university in Switzerland that's been avant-garde scientist, and Aurélie Jeanne, she's a computational scientist who has been working a lot on biases. And it's great that two power ladies are coming and sharing the experience. This time we'll talk about biases and robotics. And yeah, we'll continue the series as much as possible. We'll have another series two days afterwards on research. This is on a paper that has been co-authored by 2 of our MicroOne colleagues, Nima and Ali, but also 2 professors, Emil and Ada. And I'll be able to moderate this together with Ava, who is a colleague as well at MicroOne, on a paper called Better Together, where we were able to really see the positive impact of the use of artificial intelligence in recruitment. This is a solid paper with a significant sample that has really supported a a lot, the research. So looking forward to these two events. That said, Sasha, Sterling, with a lot of friendship, I wish you a good continuation of the day. Thank you so much for having decided to spend this time with me. I think it's just a rain check until we can do this in person. I know we have already attempted a period of time with Sasha and Sterling. Stockholm is also a great excuse to visit you. Yeah, all the best, gentlemen.

Thank you for hearing us.

Thanks for everything. All the best.

Thanks, my friends.

Bye-bye.