Designing Learning That Actually Improves Performance | Clark Quinn

Hello and welcome to the Smarter by Design podcast. I'm your host, Christopher Parsons, founder and CEO of Knowledge Architecture. And our topic today is designing learning that actually improves performance. And our guest is Clark Quinn, who is a cognitive scientist who has spent his career translating decades of learning research into practical guidance for organizations. He is the founder of Quinovation and co-founder of the L&D Accelerator. His work is grounded in a simple conviction. Most organizations are leaving enormous potential on the table, not for lack of effort or care, but because the science of how people actually learn has rarely made it into the room where learning decisions get made. In most AEC firms, learning and development didn't start with a formal strategy. It emerged organically. Executives responsible for talent came up through practice. L&D leaders stepped into their roles because they wanted to make their firms better. Not because they were trained in the discipline. Subject matter experts shared what they know without ever having been taught how to teach. As a result, most learning organizations in the AEC industry were largely built by accident rather than by design. And in that gap lies a significant opportunity to create learning that doesn't just inform, but actually improves capability and performance. That is what my conversation with Clark is all about. He walks us through the science that most accidental learning and development leaders never had access to. He explains why training so often stops at information transfer, what it really takes to design for performance rather than content delivery, and what the research says about learning design that actually moves the needle. We explore the shift from content-heavy training to practice-led training. We explore how to identify the root causes behind critical performance gaps before reaching for a training solution and how to determine whether learning is even the right intervention in the first place. We also step back and look at what a true learning ecosystem requires, not just courses, but performance support, job aids, communities of practice, mentoring, and the cultural conditions where learning compounds over time, where knowledge is shared openly, where failure is discussed, and where leadership sets the tone. Finally, we go deep on one of the most important dynamics in any AEC firm: how to effectively work with busy and highly billable subject matter experts by drawing out what they know, pairing them with skilled learning designers, and building a coaching culture that makes expertise transferable at scale. If you lead an AEC firm, build learning programs, or teach others what you know, and you've been largely figuring it out as you go, This conversation offers a foundation for doing it smarter by design.

Okay.

Clark Quinn from Quinnovation on designing learning that actually improves performance. Here we go.

If the future's got a little grime, at least you know the train will come on time.

So Clark, you have spent years helping organizations rethink learning and development through the lens of cognitive science, performance, and innovation. And one of the ideas you're especially known for is the performance ecosystem. And I really want to talk about this and dig into this today, but I kind of want to start with how you got into this field. From my understanding, you got into this through through cognitive science. And can you talk about that journey and what drew you from cognitive science into learning and development?

So I was doing tutoring on campus and got a job doing the computer support for the office doing tutoring. And I saw, I hate to tell you how long ago this was, but computer supporting learning was a relatively new idea. And, but I saw the connection, said, hey, maybe this is worth studying. And that we had a program where you could design your own major. We didn't have a major in that, of course, but we had a program where you could design your own major and I ended up designing my own major and it's been my career ever since supporting learning through technology. And it turns out technology turns out to be conversations as well as devices as well as lots of other things. And my first job out of college was designing and programming educational computer games. And that notion of engagement has remained a recurrent theme because it's relevant to learning. But I realized we didn't know enough. We were coming up with questions about how to design things. And I said, I wanna focus on, you know, well, I read an article calling for cognitive engineering, focusing on how we think, work, and learn, and designing to match how we think. And my twist on it was let's design for how people learn. And that I did PhD, I was doing the academic route for a while. And I came back to the US. I was overseas for a few years and I came back to the US and was looking for an opportunity. And I said, this makes perfect sense to connect organizations with learning needs with the technology and learning science that was what I had studied. And so I came in a back way, my, from designing and programming educational computer games and getting that focus on cognitive science to meeting with organizations, but I'd never had the traditional instructional design training. So instead I've been, I think that's remained a recurrent theme, that focus on how our brains operate and trying to design solutions that work in conjunction with that.

What, if you go back to young Clark learning, trying to do these things. No, no. What was the biggest surprise about how people learn that you didn't know when you first got started?

Well, I guess the same as for a lot of people, the thought that it really requires effort and practice. In some sense, learning came easy to me. I just, I loved reading, loved looking at diagrams. I'd sit with the, my, in front of the bookshelf with a World Book open in front of me and just look through the diagrams and understand things. So the thought that it actually took work and that in particular to have an ability to do, not just to know, but to do, really requires persistent practice. And the nuances that get quite fractal in a sense that everything unpacks more and more. But in just in short, that really, if you want to be able to do something new that you can't do now, you're going to have to practice doing it and get feedback. And so that was what I would suggest surprised me and ends up being true for a lot of people is that mismatch between So everybody's been through school pretty much, right? Right. And we think therefore that's what learning looks like. And school is actually a really bad model. We forget how little we actually learn from schooling.

Is that because it's more focused on information transfer versus like you, like what?

Yeah.

Why did we learn so little from school in your way of phrasing it? And what would be better?

School does have persistent reinforced learning, which is good. But it's focused largely on knowledge. Most of the schools we see want you to be able to know this stuff, to know history and to know science and not do anything with it. And what, in organizations, what we care about is people able to do things with that knowledge. And that, you mentioned performance ecosystem, and there's two components to that. One is the focus on performance. And that's really what differentiates, I think, perhaps school from organizational learning is we really need people to be able to do things, not just know things.

Or a traditional K-12 approach versus a vocational program, which is focused on helping you actually perform things in the world.

Sure. And those can be better or worse too. Some claim to be vocational and aren't, and other ones really are and really get it. When you look at how we traditionally learned before, schools are a relatively new invention. They're only in the past few hundred years. And it came from, there's a picture of Prussian kids, military students sitting in rows and their hands are locked in a position so they can't move and they just can listen to the information. But that's 1800s. Up until that time, we had learning, but it looked much more like apprenticeships. And John Love and Étienne Wenger talk a lot about this and the historical practice And I really like what Alan Collins and John Seely Brown talked about with cognitive apprenticeship. And they looked at different initiatives teaching reading and writing and mathematics and found some similarities that said, if you really want to develop understanding, it should look a lot like apprenticeships.

And, uh, but instead of cognitive apprenticeship, meaning instead of physical skills, right, for knowledge work skills.

So developing those skills to do reading, writing, arithmetic, and other things that increasingly our work is knowledge work. It's not physical work, right? We're getting robots that can run half marathons. We don't necessarily need to do it. And we have complicated machines to harvest crops and things. So increasingly we're in an information age and we need to do knowledge work. And that's why cognitive apprenticeship becomes a useful way to think about how we should design learning for actually being able to do mental tasks, which are largely what we do these days.

And so if the role of maybe traditional education is not necessarily to build skills, but to lay down some kind of intellectual foundation, teach you how to think, all those kind of things, is part of your, I guess what my impression, we don't know each other very well, but my impression from reading about you is you're a bit of an outsider and a kind of a critic. Of the system. And so I want to get into that critique a little bit. And what I'm inferring from this is we took that same approach to school and we just took it into the workplace and it was more information transfer and it wasn't about teaching in a way that could help change behavior, change performance, that kind of thing.

Yeah, I'll go back and challenge a wee bit your statement you made early in that perspective that school is about getting you the foundations and learning to think. And actually it doesn't do a very good job of teaching you to think. It teaches you the foundations and you do need the foundations as people like Paul Kirshner make quite clear. But I do think that we aren't doing a good enough job of developing people's ability to do. And I also think we don't do it enough, development of people's ability to think. As you suggest, that's actually what we should be doing in addition to laying those foundations is teaching you to think and expand your own learning. That's not really part of the curriculum and it's certainly not— the notion that you change the curriculum and the teachers will adapt is actually naïve. You have to equip the teachers with skills to adapt and to add these things and look at their overall perspective and just assuming that they can do this is not fair to them nor to the students. So I think we could and should be adding in the learning to think, the metacognitive, meta-learning skills. In K-12 and higher ed, but they're not there now. And so organizations, if they want people to be able to do this, they assume it without actually testing it or developing it.

Organizations assume that when new people come into their organization, they know how to think?

They know how to learn.

They know how to learn.

They can think in particular domains that they have been trained in and/or have prior experience, and you hire for that experience. But assuming that those people know how to continue to self-develop and acquire new skills is kind of naive. Now, we do have the L&D department. It is supposed to train. But their training, back to your closing point, is very much influenced by that experience with school. We also tend to have, take people who are good at the job and turn them into trainers. And they don't know how to train. And then when we have hiccups like 9/11 or COVID and everybody has to go online, we say, oh, here's an authoring tool. You're now an instructional designer. And we have a lot of people that don't understand instruction doing instruction. And that's a problem. Cami Bean wrote the book, The Accidental Instructional Designer. And that's too often the case that we have people who are expected to know learning who don't. And yet have to develop it. And so they track back to what they learned in school about how to do education. And it looks a lot like school. And it's presenting information and testing information. And people will happily recite back what they've learned. And yet we have what in cognitive science we call inert knowledge. Because they go out in the real world where that knowledge is relevant, it doesn't even get activated because they've never used it in context. And that's what I would think that L&D should care about remedying is not just people's ability to know enough stuff, but people's ability to do stuff.

You've thrown me like 3 fastballs and I'm trying to figure out which one to swing at. So I definitely want to come back. I want to talk about the thing you said about people not knowing how to learn, and I wanna talk about people not knowing how to teach. Those seem like two meaty subjects. Can we start with people not knowing how to learn? Like, how does one learn?

A good, effective self-learner should set themselves goals, should design experiments. Our colleague Harold Järke talks about personal knowledge mastery and has a workshop on it. That really digs into the skills required. And he's broken up into sort of 3 major components. So seek, how do you gather information? And it turns out there are lots of folk psychologies about each of these steps. And seek, people think, well, if I just Google, I get an answer, that's great. It's even worse now when you just get answers, not even links to articles, you just get answers and they may be made up, they may not be right. Because you're just a novice in this domain, you don't know if it's right or not. But seeking, creating a good feed, monitoring that feed to make sure you're getting the regular feed of information, and then also knowing how to write good search queries and how to evaluate your results and such. There's this myth of the digital native that says, oh, they grew up with this technology, they're good at using it. And yet when there were two separate studies done by librarians in the UK and the US, and they looked at people's ability to write good search strings and ability to evaluate the hits they got. Now, this was before AI, but even then, there was no difference by age. It depended on the individual's experience. As our colleague Jane Bothar says, she's the oldest millennial because she's a digital native, even though she's close to my age. We have a lot of myths. So beyond, so that's just seeking. And then there's making sense of the information. So sense is the second stage. And that's involved in experimenting, applying it, testing it, refining it, making sure you understand it, what you've heard that means something to you that you think is worth sharing with other people. And then the third stage is share. Who to share with, how to share. And that creates a cycle because it comes back. People read what you say and some people will comment and then that feeds back into the seek again. Are they improving? And when you do this well, you get a really productive learning cycle. But knowing all these phases and how to execute each well is not something you should assume because there's folk psychology can get in the way and have you believe. Lots of people think, for instance, you read, you highlight. That's valuable. Turns out that's really simplistic and actually not very true. There are good ways to highlight, but there are far better ways to do things than rereading. And yet too many people just will reread it and think that's valuable instead of processing it, re-representing it in different ways in terms of making inferences and testing them. Lots of things we could be doing. So that aspect of how do we be effective learners is just neglected. And there's the culture matters too. So they talk about, is it safe for you to share your ideas with other people?

Psychological safety.

If you have the Miranda organization where anything you say can and will be held against you, people aren't going to share their best ideas, right?

So that's like how to learn, correct me if I misunderstood, that's like a self-directed, self-motivated kind of learning. Is— does the kind of question you said or the statement you made around people not knowing how to learn apply when they're more in formal learning structures or in mentorship structures? Are there still deficiencies in people being able to learn in that context also?

Yeah, I was just writing a post thinking about coaching, and there's the standard sort of model of sports coaching where people know about it and say, focus on your swing. Now that you were a little low consistently, let's focus on hitting it higher across the board, whatever it is. That's very domain-specific. But there's been this notion of domain-independent coaching, which says, what did you do wrong? What should you do differently? Who might you ask? And they deliberately choose people who don't know the domain. And to me, what they're doing is essentially doing what you would do if you were a good self-efficacious learner. And so we, that's valuable, but should we be instead developing everybody's ability to be a good learner instead of just focusing on that? Now there's two senses of informal learning in what you're saying. So one is I need to learn, there may be somebody out there with the answer and I want to go find it and get that feedback. And then there's the other situation where there isn't anybody with the answer. So I talk about innovation and I talk about effective skills like brainstorming to address this. We just, as in formal learning, we don't know the answer when we start, but there isn't anybody who's gonna provide feedback for us. The world will have to do it or we'll have to test it. Some of that is, requires the same learning skills, but it requires finding ways to get the feedback from the world because we can't get it from somebody else who knows more. And so that to me is what distinguishes some of the formal learning from the informal learning. And both are, both have their role. When you're starting in a new field, you don't know what you need to know. You don't know what's important. So you need some. I'm not one of these people who says don't have courses. I say do have courses that are effective, not bad courses, but you need courses. But then you need different things as you start developing. As you move from novice to practitioner, you know what you need to know and you know what's important, you just need it. So you start needing performance support to do those things you don't do frequently and it's not worth trying to put in your head. So, well, let's put it in the world.

What's an example of performance support?

The video I watched to repair my dryer. Mm-hmm. My dryer stopped working and I couldn't figure out why. And I looked online and I found this video and it talked about diagnosing and replacing this particular piece. I don't even— I remember vaguely where it was on the dryer. I don't even remember what it was I did. And that's fine because I got the dryer fixed. It was performance support that allowed me to do that. And yet I didn't have to learn anything.

Or if I'm reviewing an NDA and I haven't done it in 6 months, but we have a checklist for clauses to look for, that's something that helps me in that flow of work.

Checklist, decision trees, process guides. There are lots of tools we use. To make sure that we don't have to put it in our head. We put it in the world so we can. But there's also times we do want to advance our understanding and then we need resources. And but the value of that, that's formal in a sense that somebody's created it as well. And the performance sport, both of those have been created, but they're for further down the learning journey. But also we start needing community, we start needing experts for mentoring and coaching, and we start needing just interacting with our colleagues to see how they did it and to share how we're doing it and see what feedback they have for us. And so that's that ecosystem part of the performance. The performance is making sure we can do it. And as novices, we need courses and then we want to start just getting job aids for when it doesn't have to be in the head and we want to start advancing what's in our head. But to do that, thinking that one tool is going to be able to do all that is naive and probably not just wrong on practical basis that no one tool should be— you should— Swiss Army knives are great when you're camping, but in the kitchen you really want all the right different tools. The same thing when you're in an organization, unless you're a really small organization, you're going to want the right tool for the job. And that means having a good portal solution, a good course solution, a good communication solution. And that's where we start talking an ecosystem of support so that all our different needs for learning are met.

Can we— let's— can maybe we can use a hypothetical to explore this performance ecosystem a little bit. So in our industry, architecture, engineering, construction, I'll just pick an example. Let's say that we have an architecture firm that focuses heavily on healthcare, which is a complicated discipline. There's all this medical equipment, there's all this jargon, there's all this compliance, there's different user groups that you have to understand— surgeons and nurses and administrators. And so if patients and patients, of course, and family and all the needs, administrative staff. So let's say you have an undergraduate in architecture and maybe you've worked on some school projects and now you get put into, you're going to be doing some healthcare projects. There's this kind of desire in our community to, to find a way to speed that learning cycle up a little bit so that I can be competent working in healthcare and understand that as quickly as possible. So I've got I have some foundation in design and understanding how buildings go together, but I don't really know this discipline. If you're just hearing that, where, how do you start thinking about a performance ecosystem to put around that emerging professional, that novice to use your term?

Well, let's start thinking, you know, so if they know buildings and design and we're talking about buildings designed for specific healthcare needs, I'd probably also want them to get a foundation in the healthcare field. Just what do they need to know? And it's not everything about the healthcare. They may not need to know about filing forms for administration or reimbursements, but they probably need to know about what are the common things people do, what are the common things people need, and who are all the different stakeholders, just what you were mentioning. And then I'd start saying, how do I— and a company, Jos Aerts and Vivien Hynien have Tulsar, which is based in Belgium, and the name is result spelled backwards, Tulsar. But they have worked with my colleague Charles Jennings and created a whole process about thinking through just those situations and saying, what is it we need? Because some of that's going to be performance support. What do you absolutely have to make sure a healthcare solution for a building looks like, includes? So there's the standard checklist for buildings in general. There's electrical and plumbing and everything. Sure. But then healthcare, there's probably got to be the right power points for all the different machinery, and there probably has to be open space for wheeling complex equipment around, etc.

So you start having multiple checklists, extra backup generators in case of earthquake or something like that. Right.

And asking people to remember that is probably not worthwhile. They may learn it over time, but you probably want to teach them how to use the checklist to make sure you get that right. And then you have that associated to other information.

So you're building kind of awareness of the resources that are available to you as part of what you're starting.

But also you want people, you want that person, make sure they take the additional healthcare class. Then when they start going in, have them apprentice with somebody so that they're watching somebody and gradually handing off some of that responsibility. But somebody is looking at their thing and that person needs to know enough about learning to know when do I stop giving them feedback and start asking them, what feedback should you receive right now? So there's this transition from, you got that wrong, you forgot to include consideration of the family, you don't have seating for the family around the table, the bedside. Okay, and I'm making this up, I'm not an expert in the domain, okay? Excuse my simplistic examples, but correct them if necessary, otherwise understand them. But at one point you start, first you start saying you've forgotten the seating for the family, but at some point you start saying, what have you forgotten? Ah, seating for the family. You're beginning to—

You mean you as the expert see they've forgotten something, but you don't give them the answer.

You make them go, you recognize that they should now have developed enough ability that if they stop and pause, they're going to recognize of when I go through, ah, yes, I remember I didn't include this. So thank you. Yes, I'm moving on. That's how they develop. And then eventually you can start giving them real design requirements. Mm-hmm. But it's that mentoring and it's also the community. So they should be seeing what other people, new people are being assigned to do.

Different projects.

The other people and see the feedback they're getting and then start giving their own feedback and getting corrected and sharing what they're doing and getting feedback. So you start building this community and again, the culture matters to make that community vibrant. People often go, oh well, if I create a community, they'll share bad ideas. There is a period of time until you achieve the ideal status, but that ideal status is the community self-correcting. If somebody says something wrong, somebody else will jump in and do that. And that happens if you're doing it right.

If you have enough senior people in the community who can help with the domain.

But also you have make it safe for them to share their good ideas and they don't feel threatened by these new people. They welcome the new generation. And if they say something wrong, somebody else will connect them. There's a lot of factors that go into making that work. It's not just easy and simple. Oh, I just— one of the best ways to find out you don't have a vibrant community is to put a social media platform in there and nobody shares and you go, oh, maybe we don't have the right culture. So, and we're probably a generation past that now, but—

Oh no, we lived through that. When we first came to business in 2009, we were a social intranet and it was, did they have the right culture? Like checklist number one, is there psychological safety? If somebody asks a naive question or says the wrong thing, what's, how does the organization respond?

Right. And it takes nurturing to get there. And that, I found out I'm not a good social media facilitator. I'm too much of an introvert that I just don't have, because it takes time to build it. It takes Kate's, it takes soliciting people. It takes make people have to continue to find value to stay in a community, to continue to be there and gradually move from a legitimate peripheral participation to a central role. So there's lots of, and I'm, no, I gather I'm preaching to the choir here.

Well, I think it's interesting. We ran a workshop with some of our leading clients 3 or 4 weeks ago, and leadership that are like— these are companies that want to be learning organizations. They want to push forward. And they described a bunch of disconnected learning programs, activities, but that weren't cohesively brought together in an ecosystem. As your term is. And I think what you're— what I'd like to spend some time on is how does that all come together so that 1 1 7 or something like that? Meaning you've got a formal course, maybe health, let's call it Health Core 101. You've got some job aids or resources that part of what you do in the course is you make people aware that here are some resources to help you recall this, or we're not going to have you memorize the checklist in class. That's not what we do in here. They'll be in community, so they'll vicariously learn through their peers, through mentors and people in different roles and different projects. They'll get feedback directly from a mentor and apprenticeship. So maybe they have a social platform where they can ask questions in an online format. They can have a system they can go search to get information. But so I think what you're— what part of the performance ecosystem, as I understand it, you're getting at is many times it starts and stops with the course and it doesn't include all those pieces as part of an integrated solution to help somebody learn. Is that correct?

Yes. And what you say is absolutely accurate. But I want to go a little bit further because you can have all the pieces and still not have a coherent ecosystem.

Great.

Because the pieces aren't linked together. And there's more. Garvin Edmondson and Gino, in a Harvard Business Review article, talked about the dimensions of a learning culture. They actually had a— they were talking about a tool they used to make this analysis. But really, just environmental factors beyond having all the pieces, parts, you also have to care about the environment. So not just tolerance of diversity, but valuing it, recognizing the value, not just saying, oh, we will— we consider new ideas, actually being open to them and testing them if there aren't principled reasons to rule them out right away. Time for reflection, to actually stop and have time. People say, "Oh, we're too busy to take time for reflection." Turns out if you take time for reflection, you're more productive than if you don't. And the last one is psychological safety. And Amy Edmondson has continued to talk about that in a variety of useful ways in organizational culture. But then they go on to talk about you also need practices. You need to have an expectation and budget and time for experimentation, not dumb experimentation. And Amy Edmondson's most recent book is on, it's on smart failure. I forget what the title is right now, but it's about basically it's being safe to fail. As long as you're doing smart experiments, because innovation, and Jennifer Mueller wrote a book, Creative Change, which is about how innovation gets stifled in organizations because they Set up whole practices to evaluate new ideas. But what they, is implicit is that they only want ones with no risk. And innovation requires risk. Learning requires risk basically because you have to try things and be willing to get it wrong. And if you aren't willing, know what you're gonna learn if it goes wrong and what you'll learn if it goes right. That's smart. And do it as low investment as possible and don't, bet the farm unless, you know, if you don't bet the farm, you're gonna lose the farm, then bet the farm. But otherwise, maybe just try this out in a safe way. And so that's the elements that go into this. And it's not, you know, everybody's firm will be different. So where they're at, what's working, what's not will be different, which is why you essentially need to do an analysis and come up with strategy for your organization to get to a viable place. And a dimension I hadn't thought a lot about is, I talked a lot about motivation and thought about it, but it turns out there's a, my colleague Matt Richter knows, studied with Rich Ryan of DC and Ryan and self-determination theory, which is a really rigorous framework for thinking about creating an environment in which people feel autonomy, competence and relatedness are the three elements of self-determination theory. And when you have all those, you get people willing to commit and willing to contribute. He's got a new book coming out called The Motivation Blueprint, which talks about this quite nicely. And, but it's just, I mentioned slightly the fractality. As you unpack it, it goes deeper and deeper, but you can make a stop and say, at this point, we're gonna work at this level and get this right. And figure that out. And it's sort of a long answer to your question, but really an ecosystem is greater than the whole sum of its parts because it's also the environment in which it's operating will help you get the best out of it. You can get positive out outcomes from each of the element, but if you really want to get that sort of quantum leap, you really need to make sure you've got the environment right and them synergized together, working together in ways that allow people to take best advantage of it.

Right. And then, so what does that mean to take best advantage? I guess in other words, performance ecosystem to what end? What is the goal of learning and development? Maybe I'll just stop there. Like, what does good look like for an organization?

Traditionally, learning and development has been about making sure you can execute the things you know you need to do. And so we create courses to develop skills. Unfortunately, we develop courses for all solutions too often instead of knowing we can use performance support. And there's certain problems that courses won't fix. I could sell solution selling, but I'm rewarded for selling product. I'm going to sell product until you change the incentives. Courses aren't going to fix that, right? You can—

so you can put me through a course about solution selling. And if all my incentives line up for me to do the other thing, then what was the point of me— the learning?

Right. So there are lots of things interfere, but ultimately we're about executing what's optimal, which is why we're too often the first part that gets cut. Because when things are tough, people go, you know, we're executing well enough. I'm not going to worry about L&D. So I argue that there's a second component to L&D that really should be a component of it, and that is to also be facilitating innovation.

Interesting.

We know we, in theory, are the folks who know learning best, and innovation, as I've suggested, is informal learning. It's because you don't know the answer when you start, when doing research, when you're doing troubleshooting, when you're doing design, you don't know the answer when you start. That's, so it's learning, it's innovation, and facilitating that, according to what we know about learning, is an opportunity for L&D. And that's the principled reason. And then there's a very pragmatic reason. If you're the, the business unit that's responsible for adapting to increasing change and helping the organization meet the ongoing needs, that suddenly becomes very strategic and central to the success of the organization.

No, I love this. So if it, especially cuz we're kind of in VUCA times right now that are very fast moving and very change oriented. If the role of L&D is only to help scale the knowledge of the things we already know, but some of the most important things that we need to know are unknown or changing very quickly, you have to be out at the edge helping to facilitate that edge and helping the organization learn it so that you can scale things as they settle down. Is that correct?

I like the way you put that. Yes, I think that's very much the case, that it's not just about doing the thing, you know, doing the things you know you need to do is just going to be the cost of entry going forward. The only sustainable differentiator is going to be the ability to adapt, and that comes from that agility, that comes from that edge, that comes from experimentation and testing and tracking what's happening. And I'm not saying L&D is responsible for it. I'm saying L&D is responsible for making sure that it's effective and optimal, leveraging what we know.

But how can L&D— how do they do that? What are the things that they do that help facilitate innovation? What's the role like in the best examples that you can think of?

To me, it's about brainstorming is something I've looked at because I was interested in what makes creativity and good design. I have cared about how we think and I've cared about learning and how we learn and I've cared about design. How do we create solutions? And I have to understand technology and a bit of business. But that issue of design gets into creative. How do we come up with new designs, better designs? And that gets into creativity. I looked at Stein, who looked at creativity and has two handbook volumes a number of years ago now. Hate to admit how long ago. But I looked at it deeply. And so there was, several years ago, there was a series of articles in places like the New York Times saying brainstorming doesn't work. And they were right, 'cause the original proposal for brainstorming put people in a room, gave 'em a problem, and asked 'em to start talking. And that turns out not to work well because you're not leveraging the diversity of thinking in the room. If you've done this right, you get, I mentioned the value of not just tolerating but valuing diversity. This is where it pays off. If you give everybody the problem but ask them to think and not come together or not share ideas until everybody's come up with their own ideas, then you share them. That's how you populate more broadly the solution space, and that increases likelihood of finding the best solution.

Or maybe even in some cases, letting people think about it for a couple of days in advance for the people that need to reflect and have their unconscious subconscious work on it.

That's even better so that there's time for that percolation and incubation and fermentation. Pick your metaphor. But that's absolutely right. We want to get those best ideas down and then we share them. And before we evaluate them, we create, hey, does that— what if we put these two together? Let's randomly populate spaces. They have apps that throw out random ideas just so like, you know, what does purple tennis shoes have to do with the solution? Well, let's figure out what that means. And then you, so you diverge first and this is classic Diamond or Double Diamond.

Double Diamond. Yeah, yeah, right.

You diverge, you get as many ideas you can, then you converge. I knew an interface design company in Australia when I was there that actually set two teams on every problem and let them run. And then they brought them together and said, okay, now what's the best parts of each? What should we take? It's— that's— it was a bit of a luxury to have two separate teams doing this, but the outputs were better than if they didn't.

Hmm. This is really an interesting frame because maybe partly this is because we work in the design and engineering field. There's a lot of those kind of thinkers per capita in our clients. But this idea that for a learning organization, I'm kind of actually adding— so I had written two things down and I have a third. So innovation. So new things, the frontier, the cutting edge, etc. Then you have a kind of a function of scaling. Once you kind of have established some things that work and you want more people to know how to do the things that the pilot team did. But then there's a phase of unlearning also where we need to retire and stop doing that thing that we scaled is no longer working, no longer accurate. There's that kind of piece to it too.

Yeah, you probably saw my reaction and that's to the term unlearning. I don't like that term because it, and as somebody pointed out, maybe Tom McDowell, that it makes it easy to think that we can just unlearn stuff. We can, and it's not. You really have to, and technically we have to learn over the old traces. And the old traces can be quite strong and recapture our behavior.

Under stress. Yeah, yeah, exactly.

Under stress, over time, we'll, backslide if we're not actively reminded and prodded and trained to make sure we now have this new response. So yeah, we have to get rid of stuff, but— and the— I like the Garvin, Edmundson, and Gino because they not only talked about experimentation, but they talked about analyzing the results of those experiments, figure out what they meant, and baking that into your own— for increasing what you're teaching and training about is the results of that learning. And yeah, you may have to If you come up with a better way to do Task A, you're going to have to learn over the old way of doing Task A. So maybe you did Process X and now Process Y is a better way to do Task A. Let's—

that's interesting though. But if Process Y is 5 or 10% better, you also have to calculate the cost of retraining your entire organization for marginal improvement. Do you actually end up positive there in terms of ROI?

Oh yeah, yeah. These are the decisions you have to make. Is the investment worth the cost? Is the output, the benefit, worth the cost it's going to be to make the change? And in the short term, 10% improvement. But if you multiply that over multiple years or products, maybe it is worth it, right? It depends on how much it costs to do it and how much it costs that, you know, what's This, and this is the classic ROI calculation. The, my only problem with ROI is sometimes it could mislead you. You could do something small that has a really big ROI and something bigger may not have as big an ROI, but it's gonna make a much bigger difference in your organization over time.

Yeah.

And you have to make sure you prioritize those relative impacts appropriately.

So can we talk about we talked about the ecosystem or performance ecosystem, but we didn't really talk about performance too much. And one of the things that you and I had talked about the first time we had a conversation is it feels like at least, and actually let me, let me tie two ideas together. You said the Accidental Learning Instructional Designer, you mentioned that book, but there's kind of a, of a level above that, which is like an accidental learning organization who kind of stumbled into— I think all organizations are learning organizations because they have to be in order to survive. Like, you couldn't keep growing and evolving, blah, blah, blah. But there's kind of doing it ad hoc, and we picked up some things and some things to remember from school, or maybe something you saw another company doing, versus there's very intentional organizations who think about measurement and success and performance and innovation, all the kind of things we're talking about. What's that journey look like from an accidental instructional designer slash learning organization to something that you would consider best in class?

Yeah, I wanna also characterize the accidental instructional learning organization. There's also the organizations led by people who think they know the answers. And we have this culture that says, oh well, if you manage to succeed and make a successful company, you're brilliant. And that they totally discount the influence of luck in that. And the research shows that luck plays a much bigger influence. But sometimes they make all the decisions for everybody. Instead of looking and tapping into, you know, it's the old hierarchical approach still. And I recently saw something on that that was talking about people at the bottom who have to think a day ahead, and people at the next level think a week ahead, a month ahead, years ahead. And it's like, we're in the information age now. And Keith Sawyer's document as an academic, and Stephen in his book Group Genius, and Stephen Johnson in his book More journalistic approach to where good ideas come from. Talk about this. That myth that one person goes away and comes back with the solution isn't true. Everybody's— the better answer is social. We are smarter than me, as the book on my shelf titles it. That idea is important. So how do you intentionally get there? It begins with figuring out how to facilitate the flow of information, the flow of communication in the organization. It's you make it safe and you have the tools so you can have that conversation. And it's also what the leader does. One of the things I added to Garvin and Edmondson's approach, it wasn't just leadership that supports it. I believe leadership has to walk the walk. So, and Jane Bozarth has her book, Show Your Work. You can't just say, oh, make your mistakes visible. You have to do it yourself as a leader or nobody's really going to believe it's safe to show your mistakes. And a small software company up in Redmond, Washington had such an aggressive culture that if you shared a mistake, you would be eviscerated for it. So they all made the, kept making the same mistake because nobody could admit to it, right? This is not a good situation. It's the, so it's a conscious decision to take advantage of the people in your organization and creating an environment in which they can participate in the decision-making. And then there's a lot of steps that go into there. But it's, as you mentioned, you can't experiment if you don't measure because you really don't know what the outcome, if it's good or bad, unless you have a measure.

You don't necessarily know what the clarity on the goal of the learning program or a specific learning, you know, activity, like what is, what needle should get moved if this goes well?

Well, and that's one of the biggest issues in L&D is measurement. As too often they measure, did people like the experience?

Right.

And there's nothing wrong with that except that its correlation with actually having any useful impact is essentially zero. Salazar et al. analyzed this and got a pro— a meta-analysis, 0.09 impact. That's zero with a rounding error. That's no correlation between whether people liked it and whether it was effective. And in fact, there's plenty of examples of the instructor who was not the most liked and yet was the most effective when they measured. And so this is—

how do you measure if learning's effective? What does that mean?

Ultimately, it's what you said. It moves a needle in the organization. You should not actually start an initiative if you don't know what it is you're trying to fix. And there's a field called performance consulting that I think essentially should be part of L&D, which goes in and says, before we create a solution, let's make sure we understand the problem. Now, what is the gap between what should be happening and what is happening? Is that gap significant that we need to remedy it? Let's look at all the gaps in the organization, figure out which ones we need to fix. And then what's the cause? So that's the gap analysis. And then there's the root cause analysis that says, Why do we have that gap? Is it wrong incentives? Courses aren't going to fix that. Performance support isn't going to fix that. We need to change the incentive program. If people just don't have all the resources they need to do the job, we have backlogs of stuff. Courses aren't going to fix that either. Now maybe it's knowledge. Ah, well, now we're beginning to find something. Is this a skill, something we need people to do that they can't do now or they're doing wrong?

Or maybe they don't feel safe and/or they don't feel safe to raise their hand if they don't know what they're doing. So now with skills, knowledge, and culture, perhaps.

And so you work backwards and you eventually say, what would people be doing differently that would lead to this result? And if people would be— and then you look to see, are people doing this differently? And then you work back and say, are we training them to do it differently? And then, so these are the measures. And but it's ultimately, what is it you need to change in the organization? If you need salespeople to be shortening the sales cycle, well, you figure out what the sales cycle shortening process looks like and you train them on that. And then you look to see, is the sales process shortening and is that yielding the increase in sales that we want, whatever it is?

But only after you validated that the problem isn't that no one wants to buy the product.

Right.

Because no amount of training on selling will help.

Well, exactly. That's performance gap. And the performance gap should be we're not selling enough product, and it turns out that the sales cycle is too long, right? So, you know, so why is the sales cycle too long? So you validate all this and then you figure out what the intervention needs to be. And sometimes when it's a course or a job aid, you— L&D steps in and develops that or curates resources. If it's a problem, that's when you assign a team where you don't know the answer. And Matt likes to point to a British historian, Keith Grint, who analyzed military, but talks about 3 sorts of types of problems. There are the tame problems where you just manage it. There are the critical problems that are time bounded. If you don't act, somebody's gonna die. Well, you just need someone to make a decision. And then there's the wicked problems, they're the ones that you have to assign a team and manage the process and figure out what are you gonna do. And different types of problems require different approaches. But knowing when you have a wicked problem, when to assign a team, and when to test out the results, and what sort of time frame do they need— in many cases, it could be unrealistic to expect a significant change to come out of a week of meetings.

So we're kind of working upstream from learning.

Yeah.

To organizations getting better at classifying and diagnosing problems correctly so that not like basically that so that the solution to every performance problem doesn't become training or learning or something like that.

Right. And while that may make training uncomfortable in the long term, if you reduce the courses you develop to the only ones where it's really needed and then you do them well, you have more resources to do the other things like facilitating innovation, like creating or curating job aids and learning resources and making sure that the community is vibrant.

Do you think learning development teams try and take on too much in terms of scope? It's kind of implicit in what you just said is that maybe they're doing too much that maybe could be culled back for some less effective ones.

I'm not— they think they're doing too much, but I think they're doing too much of the wrong stuff.

Okay.

So I think they're doing too much content development. Back to how we perceive school, if I give people new information, they're going to change their behavior. And that actually is, I believe, and I don't really have good evidence. I don't think there's contrary evidence, but I just don't really have good story, but We used to believe that we were formal logical reasoning beings, right?

And we are, we are feeling machines that happen to think, right?

Right, right. And if we— so then if that's true, we could give people new information that would change their behavior. Yeah. Oh, logical, it's— this is not right, this is right, okay, I'm going to change. But we're not. We're complex beings, as you say, feeling machines with capability to think, not thinking machines with the capability to feel. And, but it's not even explicit, but I believe it's implicit in so much of what we see in organizations. Yearly reviews. That's just been proven to be worthless. Why do we continue to do that? Even twice yearly reviews isn't frequently enough. People really need regular feedback. And the list goes on, bullet point and whatever training. Just giving people information isn't going to change their behavior. And yet too often, here's the content, PDF, PowerPoint, add a knowledge quiz and you're done. That's— and people now want to do that faster and faster. And why do you think—

I mean, we're in 2026. Like, I've heard what you just said for a few decades now. Like, why is it taking— why is this still a problem?

Why has education been recognized as ineffective for decades and it's still True. I think there are multiple reasons. L&D isn't very good at communicating and has been quite comfortable just taking orders and producing courses. Nobody cares. It does— they don't measure, so they don't know that they're not effective.

And they don't measure beyond did somebody like the course, right? Did they complete the course? That's worth it.

Well, that— it's nice if they even do that. Most people do that, but not everybody.

But what they don't measure is did this person move the needle? Did their sales cycle shorten? Did they do their door schedule faster? Did they make less? Yeah.

Even can they now do it? Are they still doing it in the workplace? And is that leading to the change we need? That's the story we should have and we don't.

You mean, can they do it more than 10 minutes after the training? You know, that's— yeah, yeah, yeah.

Oh, and there's a reliable result that if you just do training, 3 days later it's gone if you don't reactivate it. Working with a startup, Elevator 9, that's trying to develop a solution. Using learning science and technology to follow up on face-to-face courses so that learning does persist and actually transitions from not just can you recite the information or solve many scenarios, but what's happening in the workplace and are you ultimately seeing any impact change? So we don't measure, we have bad ideas about what good course design is, and our stakeholders, think that if it looks like school, it must be effective. So they're not really asking for this data. I'm still waiting for that to change, by the way. Eventually CFO's going to go, we're spending money on training. What evidence do you have that any of this is doing any impact? And I would rather L&D be changing to prepare for that rather than faced with that as a sudden and surprising outcome.

We have bad ideas about good course design. What is good course design?

Well, as I mentioned, practice. I believe strongly that we've got our ratio wrong. We tend to have about 80% content and 20% practice. And most of that's knowledge test. And there's research that shows, Pujagural showed that we used to have this belief that we need to make sure they have the information before we test them to see if they can use it. And so there were low-level questions and high-level questions, and she clearly— the final test is, can they do high-level things? And but used to believe that you need to test knowledge and then do this. Lots of people just test the knowledge. That's the pro— as I was mentioning, is a problem. But if you test the knowledge and do the high-level questions, you should transfer to here. But it turns out if you just ask high-level questions, you still get that transfer to ability to do. You don't need the low-level knowledge questions because it turns out here you have to retrieve the knowledge and apply it. Here you're just retrieving the knowledge, but ultimately you're going to have to retrieve the knowledge and apply it.

Can you give me an example of what a high-level, low-level would be in this scenario?

Yeah, let me see if I can make an architecture one. Level question would be, What are the requirements for a weight-bearing beam in terms of the current code? Okay. And you recite an answer. The current code says that a weight-bearing beam has to be this long and sustain this much weight. Mm-hmm. That doesn't mean you can design. What you then need to do is say, here's a building that has this much weight. How many bearing, how many How many columns does it need? I'm making this up 'cause I'm not an architect, but that difference in question is applying the knowledge about that weight to then say, what do people really need to make decisions on? What are the decisions they need to make in the real world? Not as, not what are the knowledge they need, but what are the decisions they make? And just asking those questions requires you to retrieve the knowledge and apply it the same as just asking the knowledge retrieval questions. So you don't need to ask those knowledge retrieval. Questions.

Is it— is partly the reason that people do these courses this— I'm sure there's a lot of underlying, but I'll ask if one of them is it's easier to test on the low-level stuff because that could be multiple choice and fill it in where the other one has to be an interactive exercise. Someone has to evaluate whether or not their design that they came with would work. Do you think that— is that part of this?

Yes, and I do think it's part of it. It turns out I recently ran for the second time, actually the third, a workshop on writing mini scenarios. You can take multiple choice questions and turn them into scenarios. So instead of asking, you ask that question about how many columns do you need, and then you have several ways people reliably go wrong, like too many, the right number, and too few. And you have specific feedback. That might seem like enough. You were probably assuming X, but you forgot to add in Y. Or, yeah, this many more would make sure, but you're not maximizing the space because you're having too many columns. And the right answer says yes, the formula says this, and that means you've rightly addressed it. So you can write better multiple choice questions. Hmm. Even with your standard multiple choice question tools, you can still do better jobs of writing mini-scenarios, but you have to know that's important. And it turns out It's not rocket science, but people can reliably go wrong. One of the things I found the first time I was trying to train people to do this is they would write decent scenarios, but then the alternatives would still be categorizing things instead of making decisions. Hmm. And so that's why I created the workshop and ran the workshop now twice. And it's, it's challenging to get people to get that message, but it's valuable. And so it's worthwhile to recognize and start thinking about. So, so the most important thing is to start making realistic practice and having a fair amount of it, because that's what makes it work.

So instead of being 80/20 content practice, you flip them?

Yeah, like 20% content, 80%. I may be exaggerating, but yeah. If you include the practice after the learning event, adding that in, it's probably about the right ratio. But it's also important, we can minimize the content. Experts go, oh, they have to know this and they have to know this. What's actually happened is experts can't tell you 70% of what they do. That's what research from University of Southern California, I mean, literally, because it's compiled away. That's the way our brain works. It compiles knowledge away. And yet, but they can't do have access to all they know. So they recite, they need to know this, they need to know this, they need to know this. And the good little instructional designer makes sure they know this and this and this and ask questions on this and this. And they release the course and nothing changes. Because what you need to do is get, ask first to say what it is they do. Now they don't have good access to this. I don't know if you remember the AI winter in the '80s.

I do.

I'm probably too young.

But I'm a history major, so I've read it all. Yeah, yes.

Right. So they built, had expert systems and they said, oh, we'll go to the experts and we'll make them and they'll do it reliably, repeatedly, it'll be wonderful. So they asked the experts what they did and they built systems that did what the experts said they did and they tested them and they failed. Yeah. They went back and said, expert, what's going on? And the expert said, I don't know. So they watched the experts and what the experts did and what they said they did didn't have any correlation at all.

Right. And from what I know about the expert system, they were so brittle because if a condition came up that they weren't trained on, they didn't know what to do. Whereas an expert, that's maybe something they knew how to solve. They just hadn't figured out how to articulate it to the system yet.

Yeah. In fact, that's why Paul Compton created his RippleDown rules was just anytime there was an exception, you add a new rule and he just, it was this cascading series of rules as opposed to an expert model. And it was closer to the subsymbolic approaches we see today that are basically all the machine learning.

So earlier you said the thing about teaching that I want to— because I feel like we're right there where people, experts aren't very good at teaching. And so in this scenario where we're trying to teach somebody about sizing the beams and they've got an expert, that's the person. And so instead of them just laundry listing everything that they think someone should know about sizing beams, what would be a better approach? Do that— does that person need to be a better teacher? Do they need to have better partners within the organization that can help them teach more effectively? You shook your head at the first one and nodded at the second one. So let's go on that.

I think it's wrong to try and force subject matter experts to become experts on learning as well. There are people who should be experts on learning and we get better results anyways if we go back to the, to the community and social learning aspect. So we should put a subject matter expert and a learner and in fact, A number of people who work with subject matter experts, and it includes Guy Wallace and the late Roger Schank and Martin— blanking on his name— from Carnegie Learning. They say bring multiple experts together and let them negotiate a shared understanding.

Absolutely.

And you just are a facilitator. And when you do that, you get a better outcome than just letting the subject matter expert teach it.

Now, if—

and on that shelf behind me—

Meaning multiple levels of experts. So like someone that's maybe an expert or kind of mid-level maybe some of those learnings are more present in their brain because they've just had recent experiences versus a more senior person. It's deeper in memory.

Yeah, I, in fact, I argue for that strongly. Don't just have the expert who understands the deep model and you're getting them to articulate the models. Not every expert, as the same group of people pointed out, not every expert is actually capable of articulating well the underlying thinking. They can do the job, but they can't articulate it well. A good expert. But then you also want their supervisors of the practitioners because they will go, you know, no matter the training, they still do this wrong and they still do that wrong. That's actually from a learning design perspective, wonderful stuff.

They see the performance gaps you were talking about before. They're aware of them.

These are the things that people, the misconceptions and the mistakes people make. And you want to have them get it wrong in the learning experience and get feedback rather than get it wrong when it matters. So hearing that is really good. And as you say, you actually want practitioners to, particularly ones who are a few months into the job, and then you go, what is it you wish you had learned that you didn't learn? And again, that gives you good input about what your learning experience should include. But experts have compiled away a lot of what they do, and they find it hard. And so they can recite knowledge, but they don't necessarily know. And they also don't understand the struggles and the transitions that novices have to go through. They don't have— because they're so far away from the struggles they had. Yeah, yeah, by and large. So that's why partnering them with somebody and having a team and having different representatives gives you those triangulating information you need and to develop it. And then not only do you need to develop the practice, and you should develop the final practice first and then work backwards until you've got practice that they can accomplish to begin with, and then you've got your learning experience. But then what are the models they need? We talk about content and we don't break it down into the elements actually that play a cognitive role. And one of them is the mental models that tell you how the world works. That says if you put weight on a beam, too much weight, and that beam's design interacts with the material it's made of, and at some point it can fail. So to make a certain safety requirement, we need a certain level of strength in every beam that meets this requirement. That's a model of how the world works. And you say, if I overload this, what hap— if I do this, this happens. If I do that, that happens. This is better than that, I'm gonna do this. That is what mental model, the reasoning about how the world works. And that comes from mental models, which are causal, conceptual, Then you need examples that show those mental models in practice. And then you need to hook people emotionally up front and deliver on that premise by giving examples in practice that learners recognize are relevant to them, pitched at the right level of challenge. There's a bunch of stuff that goes into making good learning experiences. But if you get these things right, good mental models explain how the world work, examples, a worked example or two that shows how it works in context, practice it first in simple and then more and more discriminating challenge. That's how you get, and you may have, you know, this unpack. So you might give initial models and then give it, as you give some practice, you give more complex models and more complex models, or you add complexity to the model as you go along.

So you're kind of maybe some models and examples, practice, models and examples, practice, not like, so you're just kind of moving back and forth.

Model, examples, practices. Model, examples, practices.

Yeah, yeah.

But yes, that's exactly right. And that progression is what leads to people— and my colleague got in trouble for promoting the 70-20-10 model because the numbers were too perfect, but the underlying idea is that you don't just take a course and you're done. And you think about your experience developing anything you're good at. Think of what you're good at, Christopher. You're good at interviewing people. Did you start out that way? Did you develop over time and work with people and get some feedback and improve? It was multiple practices.

For our audience that doesn't know 70-20-10, can you explain that?

That this was, it was actually research that was done at an institute in North Carolina And they asked executives, how do you, how did you end up being able to do what you're able to do? And they stripped away the stuff they couldn't intervene, and they said roughly 10% were courses, 20% was interaction with others, and 70% was learning on the job. Now, they simplified the numbers, they threw out stuff. So in some sense, people complain, oh, it's not rigorous science. Okay, but what, Charles Jennings, my colleague, really found it useful for was when executives thought courses were the only solution. He said, think about your development. Did it follow the 70-20-10 model? When they asked executives doing what you do, what they did is— and think about your ability to perform. Most of what you did, you know, how many courses did you take on interviewing people?

Zero.

Right.

Zero. Read books by interviewers, you know, and watched and listened to a ton, but actually a course? Zero.

Right. And that's too often what happens. And so that's what he was trying to help people recognize is that formal instruction can be really useful for stuff we're not biologically primed to learn. Anthony Geary talks about biologically primary and secondary. So language, for instance. We are primed to learn to speak and listen. Brains are wired to do that unless they are not wired right to begin with. But generally, most people are wired to learn. But we are not wired to learn to read and write. That actually takes instruction because that's biological.

Play guitar for most people or some— I'm sure there's tons of examples. Yes.

Yes.

That makes sense.

And that's it. And most of the stuff we need to learn, including architecture, is stuff we've artificially created and our brains aren't wired to learn.

It wasn't evolutionary, didn't exist when we were evolving.

Well, we had to create shelter, but not quite calculate building shelter with trees. And even that, if we used the wrong branches—

No cantilevering structures over. Yeah, right.

Yeah.

That's so— so it's so earlier we talked about learners learning how to learn. The teaching thing, I wanna pause, but I wanna go to, to experts in learning. But for teachers, I think all the things you said make sense to me. And then in our, this is probably true everywhere, but also those experts are highly in demand. They're highly billable, they're busy. And to, I think another reason why we're just trying to minimize the impact on them. And if they're not gonna be that effective doing it all by themselves, then having them partner helps on the business side of the whole thing as well. What, what makes for a great expert on learning? Like this person that will partner with a subject matter expert, what have they learned how to do?

It's interesting. So John Alexson did a PhD with university faculty, but on how does an instructional designer talk to a subject matter expert? And one of the important things is able to generate respect for their knowledge of the learning. And say, look, I, you're the expert in the domain, but your intuitions about how people should learn it is not to be listened to. I'm an expert in learning and making sure that's understood. Then that should be true. They really do need to understand learning. And I think everybody who designs for people should understand how people process the world. The basic human information processing loop. That's why I created a video on that. But they should also know the basics of good learning and how to— by understanding how learning happens, they understand how the compilation happens. So they know how to work with subject matter experts to get out what they need.

Meaning, are they fishing for the mental conceptual model you were talking about before?

Fishing for the model. There's a number of things they need. They need the— also the misconceptions, the ways people reliably go wrong. And why, and they need good stories about great successes and failures. And too often subject matter experts don't like to talk about their failures and they skip steps. And Alan Schoenfeld did this. He was one of the people, he was the math person, arithmetic of cognitive apprenticeship. And he would work on a problem on a board and most experts will go, oh, you do this and you do this and you do this and you get there. And it turns out most people think it makes sense, and then when they go through and try and do it themselves, they can't remember why those steps or the order of those steps and stuff. And then they can go make mistakes two ways. They may think they're stupid and they may tune out, or they may just not really understand what's going on. And what Mickey Chee did with research was showed that people who explained the steps, the reason why, did better. But experts who explain why and go, actually, because really at that first thing I could do this or this, but because of this I did this, and then I could have gone here or here. But so Alan Schoenfeld did that. And then he also even deliberately made mistakes and would continue on and then go, that can't be right because of this. Ah, I made a mistake here. Showed the unpacking of the self-monitoring.

Right, because that mental model is locked so deeply that you have to pull it out like this for somebody through a real example in order to even understand what your mental model is.

Yes. You, and so good instructional designer, when this expert says do this and then this and this, why did you do that? What else could you have done? And you know, okay, how would, if you made a mistake, how would you know you got it wrong? And you know.

So they have to be comfortable enough asking questions and not knowing the thing. And they also have to build enough trust with the expert that the expert will feel comfortable being vulnerable in front of them or talking about times they failed. And yeah, it's interesting.

And understanding that that time is necessary and valuable and structuring the organization so that they do have that time for reflection. That's a reflection activity, actually. And, you know, that yes, that they build more and they are in demand, and yet somehow you have to find a way to recognize that in the long term, if you're not giving those people time to help shape the next generation, when that knowledge walks out the door, you lose it.

Right. That, and that expert will spend that time that's going to come back to them, right? They're going to spend their— they're going to be there late at night fixing, refixing the work instead of trying to prevent it, you know, or like educate people on the front end. So they pull up mental models, exceptions, stories, including failures. Then what happens? So they figure out kind of learning strategies. Did they— is it better if they can partner with an expert, tease out the mental model stories, how they think? Is it better to still have an expert now teach? Now that the expert knows how they work, is it better to have them teach it or is it better to have another person teach it?

What, what's your— 2 reasons. One, I still don't expect the subject matter expert to become a— have become a good trainer. But also, you're taking them away from that high-value stuff, that billing. And you can do it with e-learning, or you can do it with face-to-face instruction or virtual online instruction. But you really— the instruction designer should create an experience that's going to lead to learning, you know, achieving the ability to do. They should, with good performance objectives, knowing what needs to be taught, working with the subject matter expert to make sure they understand the content. They then should design the final practice and work back to the early practice and then design the experience and test it. Because one of the things, people don't have the same wonderful predictable properties as concrete does. They— and so, and we have good learning science principles, and that designer will probably have a very good first cut. If they didn't, you would have bad first cut and it would take testing and tuning a lot longer. But they should have a pretty good first one that really you test and tune. And you look at people like Megan Torrence with her LLAMA or Michael Allen with his SAM, Successful Approximation Model. Megan Torrence's LLAMA is a lot like agile methodology, something— they're both iterative approaches. And that's what we're seeing. It's not If we build it iterative to curriculum design, iterative learning experience design, learning experience design.

So pilot it, get feedback, adjust before you scale it out and roll it to everybody.

Yes.

If you have— so if the— so if I want to go back to the idea of overly simplified numbers of the 20/80, right? 20 content, 80 actual real activities or experiences. If the expert isn't in the room or an expert isn't in the room, How— and there's— you're working through scenarios and activities and you're applying the learning. Don't you have to have someone there who can help guide whether it's good or bad if they run into a condition not expected? Like, how does that part happen?

So answering the questions, you might need what customer care centers have, two tiers. So the instructor may be able to answer the basic questions. If you've taken one of your subject matter experts, good performer, and trained them to be a good instructor, which isn't unusual, they might be able to do this. But they might say, have to say, I don't know the answer to that. Let me check it out and get back to you. And if it's an all-day training, maybe at lunch they ping somebody and ask them, what happened? Or they have to send out an email afterwards that says, okay, I found the answer to the question and it was interesting and it refined it and everybody needs to read it. But I still don't think that just because you deeply know the domain, you are going to be a good instructor in it. And therefore, you really should be a good instructor. And if you need another domain, and the more, the more upfront experience the learner has in the domain, the less that's true. So as you progress to practitioner, you really don't need formal instruction so much. You just need the ability to ask the expert and get feedback or like communities of practice become very important or that kind of thing. Yeah. I did want to make a point. The reason I was crossing my fingers is I haven't mentioned context. Okay. And one of the important things is when you're developing a skill, how broadly do you need to apply it? So if you're trying to understand designing pavilions and it could be for health or it could be for vendor displays or it could be for any other purpose.

Yeah.

You have a lot of broad transfer needed. And so you really are going to want to make sure that all the examples you see span the space of possible application so that you increase the likelihood of transferring to all the ones you are going to see. If I only teach you about healthcare pavilions and then you're asked to design a vendor pavilion, you may make huge mistakes. You're not going to transfer well. But if you've been doing pavilion design, if you're trained on a variety of different types of pavilions, You're more likely to be able to successfully design vendor prep.

Or, but conversely, you could do the other mistake, right? You could overtrain somebody if they're not ever going to do anything but healthcare work, then why are we teaching?

Right. So you need to know the space of application, then you want to pick the minimal set. And the contexts are across examples and practice, but more practice than examples. But you want to pick the minimal set. That will develop the ability but also cover the space to support the appropriate transfer. Again, you're absolutely right. If they're only doing healthcare pavilions, train them on healthcare. The variety of healthcare pavilions is going to be much more important than training them on all the different types of pavilions. Yeah. For your organization's need, they may want to move to a different position where they'll have to design other types of pavilions. But as an organization, that's not your concern. Right.

I want to loop back to the— this is sticking in my mind that I want to know what you think about this. We kind of talked about why it's challenging for a subject matter expert to do the curriculum design on their own, but we didn't really talk about why it's hard for them to train or be the instructor on that material if they had a well-designed curriculum because they partnered. What is it that makes it hard? Like, why aren't they necessarily good at this?

There's a bunch of stuff that goes into facilitating face-to-face instruction that includes reading the audience, knowing how to give good feedback in a way that isn't personal, is objective, knowing what the right elements of the appropriate feedback is. There are how to manage distractions in the classroom. If somebody's not paying attention, if somebody comes in late, if somebody's on their phone, how do you do it? There's a whole bunch of things that I wouldn't expect a subject matter expert in a domain to know to be successful. Thinking that teaching is easy is just naive. And I don't do as much classroom instruction. I've been much more computer e-learning based. My colleague Matt Richter is, he wrote a book. He worked with Thiagi, who's one of the great activity designers. Matt, they wrote a whole book on designing interactive activities that are designed to play different roles in the learning experience. But having people active and knowing how to do that instead of just presenting content is an important aspect that I wouldn't, again, expect a subject matter expert to understand. Not because they can't, it's just why would they? And some may be absolutely perfect. You know, Richard Feynman just happened to be not only a great physicist, but a great instructor. And a great writer. Not the way to bet.

Yeah, no, I mean, that's, that's a, that's a pretty good example. One of the reasons I really want to talk to you is kind of for, for majority of our listeners and community, they do find themselves as this accidental instructional designer, accidental learning organization, accidental learning and development leader, or a CEO who's leading a learning org, like all those things. What haven't we talked about that you think is really important for them to, if they want to become a modern learning organization or very successful in learning and development? What haven't we covered that's important?

There's sort of two things. One is, so they want to be an L&D learning organization.

Yep.

Great. Don't just think that it, because you want to be it, it's going to happen. You need to take concrete steps. It takes a strategy. It takes knowledge about how to do that. And the other thing is from an individual level, If I'm a— want to know about learning, how do I become a learning professional? How do I self-learn enough about learning to be able to do this? And there's lots of ways people learn. They can attend events, they can attend workshops, they can read books. You were talking about reading books on interviews and watching interviews, and all of these things are fine. Look at good design critique, but you've got to be active. You've got to be Reflective. So I talk about learning as being action and reflection. In the real world, you act in the world and you reflect on it. That's how you learn. Good instruction then is designed action and guided reflection. Now you can self-design the choice of what you're gonna do. And people say, oh, you shouldn't go to conferences or lectures. Lectures aren't learning experiences. Actually, they are. If you're a practitioner, in a day-to-day work, a lecture is a reflection opportunity to have somebody else's perspective on things that you then consciously and mentally apply to what you're doing, and it's a thinking chance. So I don't have a problem with lectures. They should be good lectures, and you should self-select as a learner which ones to attend. But a good lecture is fine. Attending conferences, reading books, But you need to take that knowledge and apply it and test it and refine it. So it's back to that model of seek, sense, share that I was talking about, about Harold Jarke. So intention isn't enough. It takes deliberate choice and action and persistence to get to where you want to be.

And I think you're kind of implicit and just buying learning technology, hiring a learning and development person. Those aren't— what I've taken away from this conversation at a very deep level is, you know, really getting to those performance gaps in the business goals and the organization and then working back to learning. Well, and also culture. Like, I mean, I think the— whether it's that the company has epistemic humility, whether it has psychological safety, I think your thing about experts where or a leader thinking just because they got to a level of success. I think the luck part's interesting, but also the times may have changed since they did it. And so there's this idea that the leader doesn't know everything and that they're, I think, connecting it to— when I think— sorry, this is the longest question in history. One of the things that I felt I watched our industry do, and I don't know if this is true outside our industry because I don't spend that much time outside of it, is that a lot of learning and development activity has been some kind of employee engagement response or surveys, feedback, employee engagement surveys saying we need more training, we need more training. And so therefore the organization creates more training because that's what people said that they wanted, needed, and that would engage them. Is that outside of our industry too? Do you see that? Oh yeah. Versus backing their way into a business strategy.

There's a lot of tactics, hiring an L&D person or buying a particular learning technology without really knowing, you know, buying the promises instead of knowing what you specifically need. There's a lot of things people do to tick a box without having it be a really focused strategy about how to achieve these ends. And lots of people will tell you, you do this and you'll have solved the problem. Because they have a vested interest in it. One of the things we did at the Learning Development Accelerator, my colleague Matt Richter and I, was created a research checklist that says, when people make claims, how do you evaluate them? You know, can you just give it a sniff test and go, oh, this isn't relevant to me? But you go down and eventually you say, where do they have a vested interest? And if not, are they following, have they done a good study? Have they got a rigorous methodology? Have they got the right subjects? Is what they've done applicable to me? There's lots of questions. About the claims, and there are lots of claims, but you have to know where you are, where you want to go, and then you can start gathering the information about how to get there. But you have to have a plan. It can't just be, oh, I'll throw some money at it, I'll throw it straight. It— like everything else, it takes a strategy, and you might hire somebody an L&D person and say, your job is to come up with a strategy. Okay, then that's their issue, but they need to follow a good strategy. And that means knowing how people learn, knowing the details of good instruction, but also knowing how— I think you need to know the basis of how we learn, why instruction works. But that gives you the basis to also start supporting the informal learning as well. Because, you know, what works and what doesn't. Why for brainstorming do you have to think on your own first before you bring everybody together? If you don't know how our brains work, that isn't going to make sense.

Yeah. Do you have a— I don't know if it's a book or like where— if I were telling you I want— yes, I want to do that. I want to be a better learning and development professional. Do you have some starter places?

Lots.

Yeah.

On my site, I have a reading list of books that depending on where you wanna go, if you want, I strongly recommend starting, everybody I think should understand the human information processing loop. That's why I have that. I think it's a half hour, 20 minutes to half an hour video that discusses it and gives you some experiments. So you actually experience the phenomena and then some explanation.

All right, we'll put both of those in the links in the show notes for the human info. Information Processing Loop video in the book list.

And then there are books that you can find. Brown, Roediger, and McDaniel wrote Make It Stick, which is a good overview of how we learn. It's a good summary of the research. I wrote Learning Science for Instructional Designers, just trying to boil down what you needed to know most about learning. It's fortunately short. The, the strategic picture, that's why I wrote Revolutionize Learning and Development. That's more than a decade old. Partly it was too early, but also that there are other things slightly more modern and more comprehensive, but I don't fully agree with the model, is J.D. Dillon's Modern Learning Ecosystem. Mm-hmm. And then Lord Niles Hoffman has The Eight Levers of EdTech Transformation that talks a bit richer about the technology picture than I did. So. I haven't found one book that I think talks about strategy perfectly. It depends on different facets and it goes on from there. I mentioned Matt Richter's Motivation Blueprint, which talks about creating an environment. I talked about Garvin Emmons and Gino's article about learning culture. I talked about Collins and Brown's Cognitive Apprenticeship as a really good model for designing learning. But conferences.

Well, let's talk about your— let's talk about LDA then, like since you're talking about conferences.

Well, that's why the LDA was created.

And what's LDA stand for?

Learning Development Accelerator. Okay. Thank you for reminding me to mention that. When COVID hit, there had been a lot of face-to-face conferences, but people weren't going to travel to face-to-face events. So Matt Richter and Will Tallheimer, another of our colleagues, big on evaluation, put together a conference on evidence-based instruction. And they were advocates. They brought in a number of us, others who cared about that and spoke and created a conference that was successful enough that they decided to create a society. So the Learning Development Accelerator is a society that accompanied the conference. We regularly have a conference now in the fall, learning science conference that we get top people and bring in to speak and we create a curriculum. And so it's not just what everybody wants to submit. We say, this is what we think needs to be covered. Now, would you talk about this? Would you talk about this? And then we now are running in the spring, it's in fact it just started, the live sessions will be in a few weeks, but, uh, you can view the CAN conference. We have a pedagogy that says view the content that's been curated for the conference and then, and discussion forums, and then attend the live sessions as well to hear, interact with the presenters live and their special sessions.

So the canned content's available in advance and there's discussions in advance. So then in the live sessions, what actually happens?

It depends on the presenter. Some want to elaborate on the material and some will reflect what has happened in the discussion forums that they want more people to see. So it, that depends, but it's interactive and it changes. We run the main sessions twice, trying to capture more of the world because it's online and we are trying to reach a broad population. And then we have special sessions that are only live that are things like we'll have a panel with Jane Botharth, David Kelley, and Henriette Cloutier. We're having a roundtable thing where you go hear 3 different short presentations in small groups with Guy Wallace, with Ingeborg Kroos, and Karl Kopp. These are people who in L&D, you should know their names if you don't.

And this one, the one in the spring is for more for people in the field practically applying L&D. And the one in the fall is more the science academic community.

The one in the fall is specifically on the learning science and how to apply it to design good courses. And the one in the spring is much more, the one that we're doing now is much more everything else you need to succeed at L&D. So there's systems thinking and what does user experience have to tell us? What does marketing have to tell us? What should you know about doing analysis right? There's a bunch of the other stuff that goes around actually designing the learning experiences. And that's a community you can join for free. There are things we do, meet the authors and some other webinars that we do free. Then there's two tiers of membership and the higher the tier, the greater discount you have on the things and the other access to certain other things you have as well. So that's, community that ideally— oh, this goes back to what you're saying, you were asking— your org needs to support people in developing. It can't just say we want you to develop, but you have to provide time and resources, whether it's the money to attend a face-to-face event, whether it's to become a member of something like the LDA. The organization has to care about people continuing to develop. And, and not only just care about it, but facilitate it, ensure that it's happening.

Yeah, that's great. I mean, I think maybe as we wind down, two things I want to ask you. One of them is you're deep in putting this spring session together. You think a lot about where learning and development going. What are you excited about in the field as you look at the next 2 years, 3 years, 5 years down the road? Like what's got you, what's got you optimistic and interested?

It depends on the time of day and phase of the moon, but my interests vary a lot. But one thing I'm interested right now is this bridging. Too often instruction, whether it's in schools or in organizations, gives you this training and stops. If you're lucky, you may get coaching or you go into an environment where coaching is expected and there are people you can talk to. That's not guaranteed. Or there may be some sort of spaced learning activity following on, but Too often we just train people up and then abandon them, and there's a big decrement in performance. And then if they're smart and the environment's not too punishing, they'll gradually acquire the skills. But we can do much more about actually facilitating that transfer from the coursework to the actual use in the workplace and the change in the workplace. And that's interesting to me. And technology is giving us tools to address this, and I think that's just an area that we could be trying to do much more systematically about ensuring that happens. And that to me is—

Is that a process thing? What does bridging look like to help take— go from a formal learning experience to actually supporting you in the work?

It depends. It could be coaching. It could be arranging mentors who are— and training them so that they know what I was talking about earlier. They know when to move from giving you specific feedback to asking you for what feedback that— what should you be giving yourself feedback on. So you're gradually scaffolding them to becoming self-improving learners. It could be spaced learning. That's like I said, that's what I'm doing with Elevator 9 is sending out little prompts over time at intervals that learning science tells us are the right intervals to reactivate the knowledge and start gradually scaffolding from what you learned in the class to applying it to the workplace to actually, are you seeing changes as a result of what you're doing? These are all opportunities that we have, we are doing idiosyncratically, but I think we should be doing systematically across the industry.

What I'm, what I'm picturing in my mind is the, is a, is a more intentional handoff between, let's call it the instructor and whoever the project leader is for the person that just went through that training. And that project leader should have awareness of what the actual learning was and what the goals were, and then understand how to basically It's almost like a continuum of care in a way. They can then take over the learning responsibilities.

Katherine Shinners told me about a company she was working for at the time where they wouldn't release training where they hadn't thought about how it was going to be extended. In other words, whether it was indeed coaching, just what you said, the project lead, so that that person knew what was happening and was prepared and was prepared to continue it as opposed to extinguishing it. Which can happen, you know. Oh, forget that stuff you learned in class. This is how we do things.

We don't do it that way on my team. Yeah. You know, that principal and one of our clients said this to me a couple of months ago. He's like, I really want all the leaders of my firm and all the leaders of our projects to understand what every single course in our catalog and all of our learning experience and all the materials on so that when I'm in the moment of need with a learner, I know exactly What— it's so obvious, but I wonder, I think it's pretty rare that people at the leadership level understand the kind of the assets in that level of detail. But if they did, my goodness, the recommendations they could make and the way they could support their people.

That underpinning that is the assumption that there's a rational structure of those resources available.

Fair.

I will regularly ask audiences, how many of you have material on your LMS that is out of date, never used, whatever? And everybody raises their hand, right? Yeah. You want to make sure that you have a rational arrangement and a curriculum of appropriate resources and includes courses, includes PDFs and PowerPoints and videos and whatever that covers people's needs. And then yes, the leader should know it, but they also should know a bit about who to recommend what, when. And you're beginning to create what I think is a sort of a coaching culture. Which is part of a learning culture, which is to be desired because each one coaches everybody else. Everybody improves if they're doing it well.

And that kind of maybe to tie a bow on this for a learning organization, that really is at the organizational level, not a learning department, right? Not a learning person, but a learning organization takes on that coaching culture.

Yeah, I do argue that L&D should master that itself first because you won't have credibility taking it out to the rest of the organization if you aren't making it work in your own organizational culture. And once you've managed to make it work and documented that it's improved things, then you have credibility taking out and going, you know, and working with early adopters and all that other smart strategy, but getting it down so that you can demonstrate that you understand what the heck you're talking about.

That's wonderful. That's a great— you have been so generous with your time, Clark. So thank you so much for being with us. If people want to get in touch with you, connect, learn more about your work, what's the best way to find you?

You can go to the LD Accelerator. I'm reachable there, clark@ldaccelerator.com. Quinovation is the way I've been assisting organizations. And I do have a blog, learnlets.com. So ideas that end up being presented in here or in presentations or in books, Show up there first. If you really need help sleeping at night, great place to go. Learnless.com.

I will put— I've been aware of your blog through us being connected on LinkedIn, and I disagree. I think it's good reading. So, Clark, once again, thank you for being here. We really appreciate you.

A pleasure for the conversation, Christopher. Thank you very much. And thank you to your audience.

Hey, thanks for listening. If you like what you heard, you can subscribe for new episodes on YouTube, Spotify, Apple Podcasts, really anywhere you get your podcasts. And if you've got comments, feedback, or ideas for future shows or guests, you can email us at smarter@knowledge-architecture.com. Thanks for listening, and we'll catch you next time.

The transportation, no matter what your station, wherever I need to be, take the BRT 5. We took the 5. to conquer our Delhi city. My favorite transportation, no matter which station, wherever I need to be, I take the BRT. Which is the best, which is the best, which is the best, it's the best. Wherever I need to be, I take the BRT. We took the fight! We took the fight!