Learning Design and AI - A Conversation with Dr Philippa Hardman

Welcome to the Future Learning Design Podcast.

So I'm seeing AI being used to build faster horses, not cars. More videos, more slides, more quickly. Auto-generate a quiz from a piece of text. I really hope this is a moment of disruption, but I don't think whether it will be is down to the technology. I think it's down to the humans who decide how to build it and what we want it for.

Hi, and welcome back to another episode of the Future Learning Design Podcast. I'm your host, Tim Logan, and the podcast is brought to you in partnership with Notosh. This week's conversation is a great follow-on from the previous episodes looking at AI in education, and I take a deep dive into this and other topics around edtech and learning design with a real expert in this field. Dr. Philippa Hartmann is a learning scientist and affiliated scholar at the University of Cambridge. She is the creator of the DOMS Learning Design Engine, which is a groundbreaking evidence-based learning design process. She's a thought leader in the world of learning design, and she has spent more than 20 years researching learning science and designing in-person, online, and hybrid learning experiences. Phil has designed some of the world's most high-impact learning experiences, including the University of Oxford's first and most successful Massive Open Online Course. She also successfully led the largest learning design project in history while VP of Learning at edtech startup Aula. More recently, Phil has started to consider the role of AI in learning design and delivery and recently delivered a TEDx talk on AI in education. You can check out Phil's Substack linked in the show notes where she shares a huge number of resources and ideas to support people interested in this area.

Hey, Phil.

Hi, Tim.

Hey, how you doing?

I'm really well. Yeah.

Nice. Well, let's, let's dive in. Thank you so much for joining. It's such a pleasure to be able to chat to you. We've connected a bit before, and I know you're doing some incredible work out there in terms of bringing learning science and bringing AI and bringing kind of that whole space of online L&D and learning to a much wider audience. So it's really impressive what you're doing. So firstly, thank you so much for joining.

And yeah, thanks for inviting me. It's great to talk with you.

So maybe we start kind of with the current moment, given the recent surge in awareness around generative AI and all of that stuff, which we, you can't get away from now. It's bloody everywhere, right? But I know you've talked before about kind of technology disrupting education and the failures that have happened time and time again with false promises around that. So I really wanted to ask you, Do you think there is something significantly different about the current moment in that whole kind of sense of thinking about the way that we might disrupt and break apart the rigid education and L&D systems that are out there?

No, it's a great question. For me, it's like the most interesting question to emerge with generative AI over the last few months. I mean, one interesting entry point for this conversation is, is that we have had AI in the world of education for like 60 years and it has failed to disrupt so far. As I would argue, all technology has failed to disrupt education so far. And so we've had, you know, the innovations that we have in education are like compared with other sectors, very, very conservative, let's say. So we went from a blackboard to an interactive whiteboard. We went from an overhead projector in my days, I'm very old.

I remember those.

Yeah.

Then we went to the PowerPoint. But what we're doing there, we were just changing the mode of delivery. We're not actually fundamentally changing the systems or the processes. And So yeah, for me, this is a super exciting question. It's like now we have generative AI and I think what's different now is like what's happened over the last 6 months is that we now have access to it. There are more people than ever with access to AI. That's the thing that's changed, not the technology itself necessarily, although there has been some changes. The biggest change is that we can all get in. It's very affordable and free. I'm assuming here, you know, I'm talking here as a very privileged British, for some people that, you know, there is an interesting conversation to be had about the digital divide, but we now have access and we have experimentation. But let's not forget that this technology and other disruptive technologies have been around for decades, have been around for as long as we've had, you know, people on the moon and an understanding of DNA. And so the interesting question for me is like, what has stopped us from innovating so far? And is that different? I think sometimes we assume that the rise of technology means disruption. I think we need to be careful about that. And I'm a historian and I always refer back to this analogy of what the comparative of the rise of the printing press in the 15th, 16th century. Of course, long term, In turn, when we look back on it, it led to change, but that change took a long time to come. And actually, in the first couple of hundred years of the printing press, what we saw was, yes, more people became literate, had more access to certain ideas, but those ideas were perpetuated. So the ideas remained the same. It's just in some ways we shored up existing ideas and existing practices rather than disrupting them. And so there is a scenario in which AI does just that. And so far, without wanting to start on a downer, but at least we can only go up from there. But so far, most of the AI innovations that I've seen in the last 3 to 6 months now that AI is cheaper and it's faster to build and all of these things. I'm seeing this same pattern. So I'm seeing AI being used to build faster horses, not cars. So it's more videos more quickly, more slides more quickly, auto-generate a quiz from a piece of text. And while these things are, you know, feel like a balm or, you know, a vitamin, a painkiller for people who have to do these things every single day, if we zoom out, you know, I have the luxury of zooming out and I think it was the Dean of Education at Stanford, Dan Schwartz, who said recently, and I was like, yes, that the biggest risk of AI is it just makes us more effective at really ineffective practice. And that's what I'm seeing so far. And so my answer is, in short, I really hope this is a moment of disruption, but I don't think whether it will be is down to the technology. I think it's down to the humans who decide how to build it and what we want it for.

Yeah, no, I completely agree. It's kind of that productivity narrative that's everywhere is just, you know, the heightened productivity. Fine, that's one thing, but there is a completely different conversation about whether it will actually change fundamental design principles or structures or ways of working, ways of approaching purpose around what learning and education is about. That's a whole different conversation. And I think it's, you know, you bring up the printing press. I think that's really interesting. And absolutely, it took— it's easy to talk about that that was a revolutionary technology, but as you say, it took decades, certainly maybe centuries to actually bring that change through. But even at the beginning, there were people like, I know Zak Stein's really interested in John Amos Comenius, who was, you know, was there thinking, wow, this is a fundamentally revolutionary technology. And out of that grew his idea about democratizing knowledge and learning through the textbook, you know, which we've inherited and we're now trying to get away from. But there were people there at that point who were thinking fundamentally differently because the technology had afforded some provocations to think differently that, okay, this might be possible. And now, as you say, there's this whole kind of open access to much more potentially democratic and distributed possibility there as well. It's, it's a really interesting moment.

Yeah, it really is. And I, I'm excited to see which direction it goes. I mean, again, history tells us that it will probably go in a number of directions, and I think we'll see some people really embracing it and innovating it, and some people trading off not doing that, trading off sticking with a more traditional content-based knowledge transfer type model. I think University of Oxford have already put a stake in the ground on that and said, that's, you know, we're not changing, we're sticking with this model and that's their strategy. So yeah, it'd be really interesting. And I think a big part of it is AI education. It's just, I think sometimes we, and it's understandable as humans, we don't like change. It's unsettling. We all have careers that are built on certain skills, certain, you know, like proficiency in certain technologies or certain processes, content. And so when something this fundamentally disruptive comes along, sometimes it's just too big. It's like, I can't eat an elephant. So maybe we just stay with what we've I've got, because I'm scared. And so a big part of what I'm trying to do is just open up a conversation about, well, what does a post-AI educator look like? What does a post-AI L&D professional look like? And the answer is not, you know, unemployed. The answer is that we can use AI in many different ways to make you more efficient. As you say, that's super important. AI has already proven its ability to increase our efficiency by 35%, and the steam engine increased our efficiency by 25%. So we're on a winner. But what does that mean? What gap does that leave? And then how do humans then just become more Well, there's an interesting question about how we use that time. So we can use that time to become more effective, to deepen our understanding, to become researchers. We'll probably have to become data analysts because we're going to be working with technologies that are generating a lot of data, but also maybe we'll just get some more leisure time or maybe we'll be more fulfilled in different ways. And so I think there's a lot of work for us to do around like looking at, I think fear is turning into curiosity, which is great. Let's start to look at what the future might look like and start to reassure people that maybe this is a, a positive, not a negative for humans.

Yeah, no, absolutely. And it, I mean, the steam engine comparison is interesting as well because there's the productivity is great, but also on the back of what, right? You know, the externalities that that created in terms of environmental damage and all of those things. It's like productivity is a really narrow goal focus. And so as you say, there's these bigger social questions about what's a good life, what's meaning in this kind of context and all those things, which are, yeah, interesting. But I, you know, I don't want to go, I don't want to get fully drawn down the rabbit hole of AI because I know, as you say, it's been around for decades and obviously there's a much more significant public awareness about it now, but you've been working in the space of, let's say, learning design, you know, e-learning, online learning, whatever name gets given to it with L&D and in education for a long time. And I'd love to spend a bit of time with you on that because I think that's really core as well to some of those kind of design principles and the ways that we actually approach this new kind of iteration of the technology, I suppose. But you were talking about Benjamin Bloom. We all know Benjamin Bloom in education for his good old taxonomy, which we love to hate. I do personally. I think there's a serious problem with a pyramid taxonomy that says one thing is better than another. But anyway, that's a whole other conversation. But I hadn't heard of his two sigma problem. And I think that's That's really an interesting thing. So this was a 1984 paper, I think. So maybe we could just spend a little bit of time saying, explaining what that is and then how does that impact your thinking around designing learning experiences in general?

Yeah, sure. And I think you're right. Like the, well, my, my mission over the last 20-odd years, I am very old. I've been doing this a long time now is, well, I was extremely privileged, went to Cambridge, spent some time at Harvard, blah, blah, blah, experienced these game-changing learning experiences and started to ask questions around like, well, why? What's the anatomy of this experience and is it scalable? So that's how I kind of sort of transferred out of higher education. I'm still affiliated at Cambridge, but I've moved out into edtech because I believe that there is a way of really leveraging technology to not just to disseminate more content or like, you know, act as a digital filing cabinet, but actually bring in some of these evidence-based principles that you know, professors at some of the best institutions or professionals down on the, you know, on the ground in corporates and people, there are pockets everywhere of people who are able to connect learning science or the science of how humans learn, the science of how they change their behavior and develop skills into really, you know, high impact experiences. And Bloom's 2 sigma problems, really interesting because in a way it just kind of captures the essence of what the ideal learning experience is. Of course, there are lots of variables and there's lots of complexities that sit underneath this, but as just a way of conceiving of the value of like an optimized learning experience and what it looks like, the Two Sigma problem effectively proved through controlled studies that the closer that we can get to a one-to-one learning experience where an expert who is really importantly two things. So they are first of all, an expert in their field. So they have great domain knowledge, but also at the same time, they are an expert coach. So they are somebody who is a great instructional coach. So they understand the science of instruction, but they're also a really great social emotional coach. They understand growth mindset. And so they're doing a lot. But the closest we can get to that as an instructor, having as close as possible to one-to-one contact with a learner means that we see significant increases in learner achievement. And so the question then becomes, well, can we use technology to capture the essence of this experience? That experience is valuable because it's personalized. It's learner-led. So every single learner gets their individual pathway to this outcome in response to their questions, the really great activities that are set in response to the coach's understanding of where they are, where they need to get to, coach's understanding of like what sort of things as simple as like whether they prefer a video or text. All of these different things, this personalization, which I think we often use, but don't often dig into, but these are the important elements of personalization, which actually drive better outcomes. And so, yeah, the, the big question for me has always been, how can we use technology to get closer to that model, to get closer to a world where we, we have direct access to like world-class expertise in subject, but also the ability for that to be taught in a way that is meaningful and tailored and drives actual real useful understanding and application for a learner regardless of who they are.

Yeah.

So yeah, that's the two sigma problem. And I think a really great way to think about the difference between a typical experience, which is usually one size fits all content and knowledge check, be that in person, online, whatever, versus this more dynamic, personalized, responsive, the sage on the stage is now an expert coach and a professor of X, and that's the the transition that we're trying to power with technology and we failed so far. But I guess a big question again to get back to AI is, is AI the key to achieving that?

No, and it's fascinating, isn't it? Because it's in-person experience. Clearly that's a really difficult thing to manage for a teacher with 30 young people in a classroom. That's a phenomenally complex dynamic that's already going on. And we know, and there's also all these other interpersonal collective things happening as well in that space. But as you were saying earlier, the kind of just creating faster horses, the technology should be able to adapt the whole process rather than just take that in-person standardized collective environment and just shift it to the online space. It should be able to do something different. But the other thing I find also really interesting, I don't know if you've heard Daniel Schmachtenberger. I've, I heard him talking about this and I think it, it's really also very interesting cuz what essentially you've got is the pre-industrial education system, which was aristocratic tutoring, right? I mean, clearly it was, it was only available to a tiny minority of people, but all of the kind of polymaths in history, like Montaigne or von Humboldt and all, all these people from history, incredibly privileged aristocracy, had expert tutors who were domain experts and who knows, potentially they understood the social emotional interaction, but that's exactly the kind of educational experience that those privileged people had access to. And so the question is, is it possible with the use of technology to democratize access to that? And I, I think that's a phenomenally important and interesting question.

Yeah, it really is. And I think we've kind of promised the delivery of that in the past. You know, we've talked about adaptive learning and adaptive platforms for a long time, but we've never been able to combine all of the things that make the optimal learning experience optimal. So that, yes, we can maybe weave a path through content in a certain order, but it isn't responsive. It's not real-time responsiveness. And it's also not supported. There are some really interesting questions, though, to be raised and some research to do. I'm doing some myself around like, well, what's the How does the student feel? What socio-emotional support? Is the learner actually engaged if they know the support is AI? Should we be telling them if it is? And, you know, etc., etc. And there's some interesting studies already out there about this and it shows there is potential. But there's also a lot of assumptions for us to test around, you know, maybe AI can technically do this, but do humans want these sorts of interactions? And, and what's the role of, for example, working with other people? We've got this real tension at the moment between the value of personalization, but also like the human need for connection. Hence cohort-based courses being really popular. So yeah, I do wonder how we will navigate all of these different technological and human needs.

Yeah, I think that's also fascinating because you've then got that whole dynamic of distributed cognition and collective intelligence. Because if we are only focused on this very narrow individualistic track of my learning and my learning pathway and my progress and, you know, fine, there's place for that. That. But there's also, as we're learning, I think more and more in cognitive science, that the need for that distributed function of groups together, wiring their minds together, you could say, um, in order to solve problems and approach challenges is phenomenally important. So where does that sit within this whole narrative of this kind of much more personalized, sometimes that's differentiated from individualized, but I think in, in some ways that is quite an individual dynamic or trajectory. So we need to be careful not to lose some of that, I think, collective import, right?

Absolutely. Yeah. And I think we will see the rise of like a new, a separation. Like, I mean, already people are experimenting with like the flipped classroom 4.0 where you do the individualized personalized bit beforehand, but then a lot of the value, and this goes back to like Eric Mazur's peer instruction model, a lot of the value comes from then comparing what you found and critiquing it and making evidence-based arguments in a collective environment that is responsive. And who knows, maybe we, we're going to be able to build AI-powered like collective environments. But I think there's definitely going to be an appetite and a need for both.

Yeah.

No, for sure. And is that what Sal Khan is doing with, I forget what he's called it, the Khanmigo?

Yeah. So, so they're, they've basically grabbed this opportunity to deliver on the two sigma problem ideal and really run with it and been really, you know, very rapidly built this beautiful machine. I think based on, they have an awful lot of data in the background to be able to train their LLM to be able to do it. But yeah. Effectively, it's a personalized AI-driven coach for both actually for students and for instructors so they can get responsive in the moment as I am working. So there's no like stop and ask. There's a chat as you're solving a maths problem or whatever it is. And yeah, that coach can get to know you, can understand where you are, can simplify things, add more complexity, record your performance. And so, yeah, that's the closest thing that we've got right now. I don't think it's actually been formally released yet, but there's a great TED Talk on it. But I think we'll see more and more tools like that. Microsoft have already released Reading Coach and Math Coach and a few other things which are designed to do similar things. So to personalize learning within a collective classroom. So yeah, it's exciting to see.

Interesting. Yeah. There's a lot, as you said, a lot of research to be done about user experience and effects and impacts and all of those things. But, but just staying maybe on the learning design part, the way that it's standardly done, obviously that's a, that's a broad statement because it's, there's loads of different contexts you might be talking about in, in organizations and businesses where they're doing learning development programs or you're doing, you classes of young people in primary school, you know, there's all sorts of different contexts, higher education, etc. But if it's possible to answer this question, you know, in general, where do you see or where have you seen the learning experience design and the learning design fall down? And obviously the flip side to that question is then your DOMS model in terms of how that responds to some of those challenges.

Yeah, and I think it's very similar to like the broad themes of what we talked about already. And so as a learning designer, I was a learning designer for many years I still am when I'm able to, because I love it. But generally your role as a learning designer is to take what already exists, which is often a PowerPoint and then some sort of knowledge check and to digitize it. And what happens then is that, and this is not the fault of the learning designer and it's not the fault of the mode. The problem with this and the reason that most learning experiences fail, or at least are not optimized, is because of the fundamental pedagogy. So that knowledge transfer, here's some content from my head, I'm not really sure who's you are as my user, but I'm going to give you all my information because that's kind of how we've been trained, isn't it? I mean, that's like how I taught when I was teaching at university is like, well, here's all of my things now regurgitate them back and I give you an A if you remember everything. I mean, it's slightly more complicated than that, but it's, that's the model. And so I think learning design processes fall down when we, we just basically change modes of delivery rather than thinking of what's the best fit pedagogy. Now the reason that happens is very, very understandable because my word, learning science is complicated. It's written in academic code, sitting in ivory towers. It's, it's behind paywalls. So even now with AI, when we can summarize research, we still have to pay a shedload for it. So it's almost designed to be inaccessible. But then also what I've found in my research is that translating learning science into design practices is another specialism in itself. That's almost like master's, PhD level skills to be able to go, okay, Here's 3,000 articles. What does that mean for like, you know, how I write objectives and et cetera, et cetera. So I think most learning designs are not optimized because they don't, they address the wrong problem in a way. Like they try to improve outcomes by changing delivery, which, which maybe increases access to the thing. But the real problem that we face as an industry, and this is L&D, HE, K-12, you name it, is that we're focused on the wrong problem. The problem is a design problem. And the big question is how do we democratize access to learning science and translate it into practices? And so that's why I created the DOMS framework, which is, it's about 300 to 400 now pieces of learning science that are most compelling, most cited into a very practical number of steps that says, okay, at this stage of the process, ask these questions, generate this data, and then use this rubric to make design decisions. And the results that we've seen are fantastic. So they, there's some some main areas of focus. One is designed for motivation. So there's a lot of learning science about intrinsic motivation, how to optimize for that baked in. The other one is how to design for memorization. So some very specific strategies for making sure that we understand some of the basic concepts. And then we progress to conceptual understanding and then finally mastery of knowledge and skills if we want to push it that far. So it really just breaks things down. It's, it's, and what we've seen is a significant increase. So we get like 96% completion rates. We saw We trialled this at the University of Coventry a couple of years ago and huge project. I think it was 1,200 modules and, you know, significant increases in achievement, student satisfaction, engagement, completion, all of the, all of the good things. So yeah, hopefully it just helps people to, you know, instead of having to go and do an M.Ed., which even then, you know, becomes outdated within a year because things change. The idea is that it just bakes in not just the what to do, but how and why to do it at different stages. Languages inclusive.

That's great because it is, I totally agree that gap, the translation gap is a huge issue because like, I mean, just as an anecdotal example, I've talked to Dan Willingham on the podcast before and he just as a case in point is very reluctant given all the research he's done and all of the, you know, he's a serious authority on certain aspects of memory and cognition, et cetera. And reading. Very reluctant to say, now here's what you should do in a classroom because there's this translation gap of research and practice. And, and I agree. I know, I don't know if you saw the OECD have just launched the Schools Plus Network for promoting a bit more of that research and practice kind of conversation across the schools network as drivers for innovation. And there's such a gap there. There's such a, a need for that kind of translation. So yeah, no, No, really interesting work. And just to dig in a bit more deeply, just could you say a little bit about what DOMS stands for and why?

Yes, that might have been helpful. Yes, sure. So DOMS is an acronym because you can't have a product without an acronym. Of course. So it's a 4-step simple process. First step being discovery. So this is all about making sure that it's learner-centered. It's all about saying, what's the pain we're trying to solve? It's kind of, I've adopted here some of the research around, it's not just learning science research. There's a lot of psychology research, a lot of marketing, but it's about getting into the learner's shoes. One of the big ways that learning experiences fail right now is that we design the course that we want to design, not the course that the learner needs. And so there's a lot of analysis. This is the A of analysis, but it is much more thorough. I would say it's very clear on exactly what sort of question, what sort of data you need in order to be able to design. And then from there, these are all sequential. We then take that data and use that for the O, which is objectives. So how to write brilliant learner-centered objectives that we know drive both motivation and mastery. So there's a science of objectives. We then used to move on to mapping because what we found, and this again is a bit, a bit of a blind spot, I think, generally, but what we found in the research is that if we think about the horizontal journey through an experience, that's one of the highest value or like most powerful levers that we have to achieve really great outcomes. And I think often we think on a kind of module by module basis. So it's like, okay, we'll do the basics now, then this, but actually what's the journey horizontally? And so in the mapping phase of Domains, we use all of the research that we have around things like the power of increasing challenge and complexity, interleaving, sequencing, all of those things that people will be familiar with to design horizontally as well as vertically through the experience. And then S is storyboarding. So that's when we get to the detail of like, okay, what's the optimal experience through a module and across the course? And there we hit principles around like minimal viable content. One of the big changes that I always recommend is shift from an 80% content, 20% activity mindset to a 20% content, 80% activity mindset, which like makes everybody panic, but it it works beautifully. So yeah, that's Dom's. And so what you come out with at the end is you can be confident that you have an evidence-based approach. I think a lot— what I've heard a lot from L&D professionals, instructional designers, HR folks, academics is, I don't really know, like I have a feeling that my design is good, but how do I know? How can I be confident? And as a learning designer, how can I go into a room with confidence and tell a professor that actually the way they do this is not optimal, but the way— but this way is. And so it gives everybody clarity, but also confidence to think in new ways. And yeah, it's so far so good. I've been testing it now for a number of years. I have a boot camp and people are coming through and I'm evolving it all the time. But we're seeing significant increases again in, you know, completion, achievement, satisfaction, and also just the satisfaction of the designers themselves, who I think have been quite frustrated historically by, you know, being seen as the person that makes the slides, not the person who is a, you know, a professional and an expert in how humans learn. Yeah.

Interesting. And yeah, I'd love to dig in if I can, a few of those sections. So one of them recently, I had an amazing conversation with Mary Hill and Immodino Yang and Zach Stein. And one of the big conclusions that they drew and they are drawing in their work is that education, and they're primarily focused on K-12 2012, to be fair, but in general, that learning is not the objective. Like learning is the means, human development and increasing complexification of human development is the, should be the objective of education broadly construed, right? Not necessarily just formal education, which I think is, that's phenomenal. So I mean, just on your point about outcomes, it's really difficult to define outcomes of human development, right? It's so like, you know, and then we get into trouble because then we start drawing hierarchies and ladders about who, you know, what kind of person is a better kind of person and all of this kind of, this very dangerous minefield. What would you say in relation to the spaces where it's actually quite difficult to define outcomes? Because obviously that's a core part of your process. How does that fit in? I don't know.

No, it's really interesting. And it is a, so this is what we tackle in the discovery part first. And we use a framework that I developed called the Learner Transformation Framework. And it's, and it's really a, an exercise in empathy. And so of course the learning experience, we have some goals that are academic that are ours as educators that we need to bring to this. So we need them to be able to write the essay or pass the test or whatever it is. But then when it comes to, it's more than that. And this is where the question of intrinsic motivation and value for the learner comes in. And so yeah, this learner transformation framework, we start with like the status quo, like where the learner currently is and the endpoint is where they want to be. And that's not just like at the end of the course, it's like in their life, what are their professional goals, their personal goals? What would fulfillment be for this individual? And then there's this gap in the middle and it's like, okay, how is your, of course your course is not gonna change their life overnight, but how does it fit in this narrative? And how do we try to not just deliver, you know, knowledge or like domain knowledge for you to give back to me, but how do we develop some of these skills? It might be things like, you know, leadership, communication. We tend to focus on those those very high-value skills, you know, so-called 21st century skills that will help to get folks to, you know, fulfillment, happiness, wellbeing, whatever. So yeah, it's challenging, but we do, I mean, one big takeaway is the value of user research. So hopefully the people who like graduate from Dom's go away and realize how valuable it is to bring learners into the conversation. And so as part of this learner transformation, part of the analysis, we bring them in and we test them and we ask them, you know, what makes you do this? Why would you bother? And then we kind of weave that into the way that we design the objectives, but then also the activities. So you can imagine if somebody really wants to develop their leadership skills, we can design activities which are about something else, but which require, I don't know, an individual to step up to lead a group to complete a project or whatever it is. So we have these two flavors, if you like, of objectives and then these outcomes. And the objectives are academic, but the outcomes are about about the user, and it's massively powerful for motivation and for achievement.

That's great. Yeah, because it is, it's really positive to have kind of a situated context because otherwise you do get kind of slightly pulled back into that just knowledge delivery paradigm again, right? Like if it's situated in this broader, as you say, kind of sense of fulfillment or development or whatever language you want to give to that, there's hopefully a pull in a slightly different direction. To make it a bit more of a significant experience. Yeah, interesting. And the other question I wanted to ask you just about the storyboarding, how much is that idea of story intentional, you know, in terms of kind of user experience? And because, you know, we know at a pretty fundamental level we are social beings and story and narrative has been a really core technology, could say, in our evolution since, you know, since we evolved language in some ways. Is that part of it? Have you called it storyboarding for a reason, or is that just a kind of a way to frame out the journey?

Yeah, I mean, partly it is just familiar language. It kind of articulates that the storyboard is the end of the process, not the beginning, which is often the case, another way that the learning design process is broken. So partly it was about that, it was about articulating how this is different and the value of the pre-storyboard bit. So let's not just jump straight to what's the video. Yeah, let's do all of these little stages. And actually, I spend 70% of my design time on these things before we get anywhere near the storyboard. So it was kind of deliberate there. But I think you're right, this concept of story and of growth and of speaking to the learner in a way that's meaningful as they go through the experience is so critically important. And that's partly what the learner transformation story is about. It's about writing this story of where you're going to start and where you're going to end and how this is going to help you along the way. And part of the storyboarding process is to make sure that that narrative is weaved in throughout. So, excuse me, at the end of each module, whatever, we might refer back to this journey. We might visualize it, but also talk about it. We'll have like, you know, moments of metacognition where we're like, you know, this is where, this is how you're doing. So yeah, there's a, there's a, the story of the course should not just be the story of, right, you've passed that test and that one and that one. It's this narrative around like what this means for the individual that really optimizes for motivation. And motivation, I think, is so underrated, partly because again, it's just complicated. But without intrinsic motivation, there's a cap on anything that we can achieve with learners in terms of their outcomes. So getting an understanding of what intrinsically will motivate them. So it's not like we're going to punish you if you don't, or you've paid X amounts, of course you will, which I think we often do in higher ed. It's like, well, you've paid 15 grand for this master's, so you're going to show up. And if you don't, well, but if we start to think about, well, what's going to motivate, and I think this is challenging. I think things are changing though, because I noticed, for example, at the end of last year, there was a the first ever academic, I think, was fired from New York University for not listening to their students who were saying, this feels irrelevant, feels like you've not pitched it right. It's not helping us. And interestingly, the institution sided with the students. And so I think there's more pressure in this direction, like a shift of emphasis from what the educator wants to teach to what the learners want to learn. But yeah, this concept of story, I think, will play a key part in that, in articulating back to the learner that I do care. I know you, I hear you, I know what you need, and I'm trying to help you.

Yeah, no, it is. I totally agree about the motivation point. I think it's vastly under-understood and underrated, and the assumption that extrinsic motivations are sufficient. And in fact, they're so insufficient that actually they completely undermine often the intrinsic motivation. And so really focusing in on that and trying to get away almost entirely if we can from that sense sense of, well, you're doing this because you've paid or you want the grade or, you know, it's going to be your passport to the next stage of your— all of that is, is really undermining for the intrinsic part, I think.

Yeah, absolutely. And I think if you look at the data, it's compelling. Without intrinsic motivation, you can design a brilliant experience over here, but it's just not— if you're not intrinsically motivated, there's only so much you can engage, there's only so much you will achieve. And so it's, for me, I always say to the people on bootcamp, it's like, if you only have time to focus on one area of this, make it intrinsic motivation, because without intrinsic motivation, everything else is kind of futile. So get to know who your learners are and what's motivating them. And this is a huge shift. I mean, it's particularly a huge shift for academics who come on the course and it's like, oh, good grief. Like I thought I just had to kind of give back all the stuff that I've learned over the last week. It's like, no, I now have to become like a product designer and it's effort and you have to care.

You mean I have to care about my students? I care about these people in front of me? Oh my goodness. Heaven forbid.

I don't even know what their names are. They're just like heads on a bench.

Half asleep at 9 o'clock in a lecture hall. Yeah. So then just to kind of maybe come back slightly to where we started as a final question with the disruption of the potential disruption of AI. And one of the things you mentioned was that idea of, or you intimated anyway, the AI replacing some of the roles that we have in society, some of the more automatable tasks, etc. Learning design potentially is one of those, right? And we're seeing more and more platforms out there that profess to be doing learning design automatically. So you just put in, you know, your whatever, your content, and it will convert this thing into a learning experience. You know, what's happening for you, or how hopeful are you around that idea of weaving in the Doms evidence-based approach into some of that? Because there is a, there's a potential that the horse bolts very quickly, and the way that those algorithms and those structures are being set up bears very little resemblance to what you, all the amazing work you've done with Dom's, right?

Yeah, absolutely. And so far, unfortunately, in the world of learning design, education, like AI ed tools that have been built over the last few months since AI has got cheaper, easier, you know, we're seeing more, more things being built every week. We are automating processes, which of course are attractive because, you know, educators are stressed people. People, they have a lot of stuff to do. The process of learning design is painfully slow. I think on average it takes 9 months to develop like one piece of learning across HE and L&D, which is just insane because by the time that you've done it, you've spent X amount of resource and it's irrelevant already. And even if that was, you know, 9 weeks, it's too long. So I'm really like not surprised that we're finding these tools and there are lots of tools out there that's like, okay, give us a URL or give us a document and we will turn that content in a quiz, for example. Soon we will, you know, we're not too far away from being able to enter text and get video and then a quiz. We can use tools like Synthesia and, you know, to reproduce ourselves and automate stuff. But I think what we're doing is we're creating a new problem that's going to hit us down the line, which is that we're just proliferating content and we are not changing the practices. And so maybe that's fine. Like maybe that's just what we will do with AI. That's what we've done with technology and education so far is we've just digitized what we already do. To do, you know, the whiteboard. But, but yeah, my, I think there is something much more fundamental that we could do. We could, you know, the Ford analogy again, we could not focus on faster horses. We could think about what's the car, the car in this instance being like empowering people to be able to design experiences which are optimized for the outcomes for the student. Yeah. For the change that you want to see the subject based on the learning science, because there is this, like you said, like this black box, there's a knowledge bank right there. We know the answer. We know what it needs to be, but it's just, there's a gap in that translation. So it's almost like we need an AI translator and that's what I was going to say. Yeah.

I mean, is it automatable in your view? I mean, is it something that actually needs humans to really sit with and understand? And as you said, kind of, you know, care about the learner and understand the full gamut of what's going on in the complex dynamics, or is it something that is automatable?

I mean, I don't think AI can fully automate anything. I think what AI can do in this, let's say in this context is like, there are patterns. This is what I've been working on now for a good few months is that there are patterns. And so, uh, you know, there's a flow chart where it's like, okay, the learner is, I don't know, this level of competency. The topic is this, the outcome, intended outcome is this. And it might be, for example, recall, conceptual understanding, or like the ability to apply it, or maybe even the ability to think originally and innovate. On this topic. And depending on like the flavor of the outcome, there are rules that we can follow in terms of like, right, okay, so the content should be this, the activity should be this, the feedback should be like this, the coaching style should be this. So they are the patterns that I'm looking at. They're the patterns that I've used to build Dom's and now the patterns I'm looking at to think about Dom's AI. But there's always like, this is true of any AI, like it's only as powerful as the humans who manage it. And so as I'm testing the AI at the moment, of course it's getting more intelligent and it's But it, what it does as a learning designer is it frees us up then to look at the output and go, okay, well, yeah, I like that, but I'll change this, I'll change this, I'll change this, I'll tailor it more to my students. And so it's, it's taking away an amount of the admin and the complexity so that I can make sure that this is optimized. But it's, I would never envisage a world where this is a fully automated, spit it out at the end type process. I think it's going to be something that almost like liberates learning designers and educators to deliver what they really want to deliver, which is the best possible outcomes for their learners in a way that can make them confident. Like I've seen with the manual process, it makes them confident, but also they bring their own character to it. And that personalization and that understanding of who we're working with is really important. And also their views on like, well, what's the best content? Because, you know, we might have ingested expert content over here, but we need the person right on the ground in the organization, for example, to say, well, these are in line with our culture, or these are things that we internally to be the most important. So it's as ever with AI, it's like a team, like AI is great, but it needs you.

Yeah, it's the Copilot kind of idea. Yeah, no, and it is, I think you're so many things there that are so important, that kind of relevance to context, which AI will be almost by definition context-free. And that's so important. And just that idea that it frees you to do more of that quality crafting, which I think I think is again, maybe it's slightly back to that productivity narrative we talked about at the beginning, but because of those automatable aspects, it allows you to augment and provide the quality through some of that crafting based on your own expertise of what, you know, you know, will make a good experience.

Absolutely. And you can imagine a scenario in which, you know, AI tells you that for this outcome, the best instructional strategy is this. And you're like, well, actually, no, I would rather apply more of a problem-based approach because I think that's a better approach, whatever. You know, would then be able to work with the machine to then optimize it for that particular instructional strategy. Because often I work with people and I can tell them until I'm blue in the face that this is the best instructional strategy, but for whatever reason they want, for example, an in-person event or a project-based thing or whatever. So it is always a collaboration and it's really important, I think, that we build tools with that in mind. So not just inputting something and spitting something out at the end. I mean, that's kind of magical. But there has to be an opportunity to be part of that production process for the human.

Yeah.

Amazing. This is great. Thank you so much, Phil. It's, I mean, it's such a pleasure to chat and you're doing some phenomenal work. And I totally, I think, as I said, that translation piece where you're adding so much value is such a huge need. So yeah, no, I really appreciate the chance to chat.

Well, no, thanks, Tim. And yeah, keep your eyes peeled for Dom's AI. I mean, at the moment it's just scrappy ideas on paper and a very basic prototype, but it's, yeah, we're testing it and it's going Well, so looking forward to it.

Yeah, well, maybe we can have a follow-up when once that's out in the world and we can talk about PhilBot.

PhilBot, yeah, TBC. But yeah, thank you. This was great.

Amazing. We hope you've enjoyed this episode. Please feel free to continue the dialogues with our guests, with us on our blog or on social media, or within your own networks.