Maximizing the Value of Learning & Development Investments | Chris + Susan of Knowledge Architecture

Hello and welcome to the Smarter by Design podcast. I'm your host, Christopher Parsons, founder and CEO at Knowledge Architecture, and I'm joined on this episode by Susan Strom, our Chief Client Officer. Our topic today, maximizing ROI from learning and development investments. So in this episode, Susan and I discuss the Modern Learning Organization Pipeline, a framework for helping AEC firms prioritize and maximize the return on their learning and development investments. For decades, most AEC learning programs have relied on familiar formats, lunch and learns, live training sessions, and recorded presentations. But now a new generation of tools like AI search, modern intranets, modern learning management systems, and knowledge agents are dramatically increasing the potential ROI of learning assets. When knowledge can be searched instantly, accessed on demand, revisited whenever somebody needs it, Learning assets become far more valuable than they used to be, but this shift creates a new challenge. How do firms decide where to invest their time and energy? You can't manage all the knowledge in your firm. So then the real question becomes, which knowledge and learning investments produce the greatest return? In this conversation, Susan and I walk through how to identify, prioritize, and design high-impact learning experiences in AEC firms using the Modern Learning Organization Pipeline. Along the way, we explore several topics like why AEC firms need to evolve into modern learning organizations. We talk about the Desire Framework, a practical tool for prioritizing learning opportunities. We talk about why learning experiences should increasingly be treated like products and how firms are modernizing learning experiences for the AI era. We talk about how learning content can become searchable organizational knowledge and why learner involvement and piloting are essential to good learning design. And finally, we talk about the rise of dedicated knowledge and learning roles inside AEC firms. Underlying all of this discussion is a broader idea. The industry is entering a platform shift in how knowledge, learning, and expertise are developed and distributed. As AI-powered knowledge and learning platforms like Synthesis make knowledge more accessible and reusable, The potential return on learning investments is rising dramatically, but the challenge for firms is where to invest their time and energy to create the biggest impact. So if you're leading an AEC firm and you're wondering where to invest in learning, knowledge, and capability building, this episode introduces a practical framework for prioritizing the opportunities that will matter most. Okay. Maximizing ROI from learning and development investments. Here we go.

Hi, I'm Susan Strom. I'm the Chief Client Officer Knowledge Architecture.

And hey, I'm Christopher Parsons, founder and CEO at Knowledge Architecture.

Chris published something in our Smarter by Design newsletter, issue 15, like 3 weeks ago called the Modern Learning Organization Pipeline. And we're gonna explore that today. So Chris, I thought it'd be good to start with just like, what is a modern learning organization? Like, what is this pipeline thing?

Yeah, so. I think maybe it's good to kind of pull back a little bit. We have been observing that there are some challenges and opportunities that, that our clients in AEC are facing. On the challenges side, we see obviously baby boomers retiring. We see really fast-paced change in terms of knowledge. So the half-life of knowledge is shrinking. There's new technology, new standards, new materials, new codes. Your clients are expecting you to do more with less. And we've got this kind of emerging generation, whether it's millennials and kind of Gen Z, who are really ambitious and want to kind of advance their career. And also there's all this new technology. So AI has hit in a major way, and there are all changes to video technology too that we've been seeing. So it's easier to capture video and edit video and clip it out and use that as a knowledge sharing technology. And so this confluence of some of these structural, cultural demographic changes, as well as the technology changes, have really created this perfect storm, but kind of in a good way to really level up the way that AEC firms are doing learning and development. So there's an opportunity to deliver the right knowledge to the right person in the flow of work, much more than there ever has been. And what's exciting about that is partly due to all that change and also the retirement slash new generation coming up. There's this desire to upskill people more quickly, to take on more responsibility earlier in their careers than they ever have. That's a career ambition for people coming up, but that's also a business reality for firms because there aren't enough of my generation. I'm a Gen Xer. There weren't as many of us born as there are boomers and millennials and Gen Zs. And then a lot of us left the industry after the dot-com bust and then after the housing crisis. So there's this hole of 15 to 25 years of experienced professionals that AEC firms are needing to figure out how to work with. And so they have to do it. And then the technologies have converged to make that possible. And so what we have built is a technology platform that combines intranet, learning management, AI search, knowledge agents, really advanced use of video technology to help enable this kind of change. That's a modern learning organization platform. But in order to take advantage of that new technology, what we're seeing is that firms have to change some of their processes and their culture and the way that they're just operating in order to take advantage of all of this new technology and to meet the moment. So I've been calling that modern learning organizations for about a year. We have been in various betas with these new technologies, like our learning management system. Susan, you know, you've been there with me since the beginning. At the end of last June, we're working with 56 firms adopting Synthesis LMS. But most of what we're seeing is that firms are working on the people process culture stuff more than the technology piece. In order to deliver high-quality on-demand learning, in order to get the kind of coverage on the topics that are important to us so we can upskill people They're needing to change their approach to learning and development and their approach to knowledge management. And so that, that's at a high level what I see firms doing. And so as a modern learning organization, they will create and maintain collective intelligence. They will leverage technology to distribute the right knowledge to the right person at the right time. And this will make them more adaptable and more resilient as business conditions and practice conditions continue to change. So that's the high-level overview of what we're seeing.

Something that clicked for me in this that hasn't clicked before is that the transition to developing the kind of institutional knowledge and then distribute that institutional knowledge effectively using the tools we have today is kind of like the transition from maybe traditional marketing to marketing today, where it's like producing short videos, producing podcasts. Taking knowledge from inside the firm and sharing it with your clients in a way that feels accessible and interesting and meets clients where they are versus where you want them to be, which is listen to your lecture. They're not gonna do that, and we just accept that. But marketing organizations have had to change the way they develop content and distribute content and manage the content. And I feel like the modern learning organization is a similar version for that inside the firm, but has a bunch of parallels. Big picture-wise, but also just some like tool parallels. You need to learn how to produce video of high enough quality, which I know we're going to get into when we talk about designing and building a bit.

No, I think that's a great— I hadn't, you know what, I hadn't connected those two things either. And what I like about it is a lot of the times what we're seeing in the betas we've been in together is that firms that— so some of them already have that marketing capability and they're turning around and they're using it to be really good at creating internal compelling content. And part of what happens there is just like you would outside the firm. You don't generally ask experts to work alone to create really compelling marketing pieces. They partner with people that are writers, video editors, like blah, blah, blah, blah, blah. Same thing we're seeing internally. But once you— so we've seen that one of our clients is doing a series where they're kind of going through some of their most senior, most subject matter experts doing these deep high-quality interviews, breaking it down into small shorter videos, creating those as lessons. But they're also seeing that as a marketing material they're creating at the same time. So these same things can be repurposed and told outside the story. The same thing, LS3P's Expert Hours. They spoke at K Connect last year about interviewing their experts internally, but that throws off those internal interviews that they do is kind of like live Teams calls throw off internal knowledge assets, but they also throw off potential marketing assets or ideas for things that can be turned into pieces. And so Yeah, I think they're not only parallels. I think done right, these things can be integrated too.

Yeah. So I'm going to take us into the modern learning organization pipeline, which is on our website. There'll be a link in the show notes. It's kind of linear. It has a few steps and then a jump and then kind of another set of steps and then jump in the next set of steps. And it starts, actually, I'm not going to say where it starts, but I'm going to say where I think a lot of firms want to start or people, humans want to start and like I may or may not be one of those humans depending on the topic, is with design and build.

Hmm.

It's like, great, I'm gonna create a course, or we wanna create a course. We know that there's this new tool set and generally I know onboarding should be better, so I'm just gonna create an onboarding course, see what it's like. I'm gonna get in design build. And actually before that I might be like, and I need to set up a recording studio because I want this to be high quality and I want it to be in short snippets cuz I want it to be. Accessible by an AI, and I want it to be modern and meet my new employees where they are. I don't want them to— I want them to have a good experience. So I'm gonna like get the right lights, I'm gonna write a script, I'm going to practice a bunch, you know, and then I'm going to record this thing. So it's not only design-build, but it's also raising the bar while you're doing design-build. So I know I, I've talked with some of our clients who've started there. Yeah, I'm curious what your thought is on starting there versus starting, you know, more at the beginning of pipeline?

Yeah, I think whenever you have a platform shift like this, and I do consider this a platform shift and a paradigm shift, there's always the kind of temptation or maybe even habit to go towards step 10, you know, versus starting at the beginning because you see the promise. And so, you know, in kind of a traditional— this may be a bit unfair, but like for a traditional kind of learning organization in AEC, the root, most atomic level of learning and development right now is the hour-long lunch and learn generally, or maybe an hour-long training session. Maybe it's not specifically a lunch and learn, but there's this hour thing that's delivered live or over Teams or Zoom or whatever. And then it gets recorded and it gets put on the internet or on a server depending on what technology you have. And that's kind of it. It's kind of like if you were there live, cool. We'll give you some credits, and then if you missed it, you can go watch the recording. Or if you come 3 months later, hey, you should go watch this recording. The thing we hear again and again from our clients is those generally just don't really get seen, or they don't have the analytics to know if they get seen. Or maybe that hour-long recorded thing that was meant to be live isn't a really great learning experience for somebody coming to it after the fact. So that's kind of where people are starting. We show them the technology that we built around Synthesis. And it's like, well, it's, you know, you can have these courses which have short lessons. Those courses can be assigned, people can discover them. We have full analytics to understand if people consume them or not. There's feedback built directly in. And oh, by the way, all that content is searchable in AI Search. So if someone just has a question about how to do a thing in Revit or about a sustainability guideline, it can just answer their questions reciting that course without you having to watch the whole hour-long thing. So there's this completely different paradigm that's become possible. And so then people want to run towards that and they think, first of all, we want to get there fast. But second of all, it's like we turn on the lights and it's like, oh wow, these existing ways we're doing it aren't going to meet the moment. And so we need to level up the quality. We've got all these old things, you know, and it's just, it can become overwhelming to the point like, Oh shoot, there's so much work to do. What started off as really exciting has now become dispiriting. Mm-hmm. And so I think you and I have seen that. And, and you know what I think we tried to redirect people towards is like, look, this is a journey. You're not gonna get there in 6 months. I mean, since you and I have been at Knowledge Architecture, we've always said you can't manage all the knowledge in your firm. There's too much of it. And so you have to prioritize. And what, I tried to do with the pipeline is help people go through a structured process to think about how to even identify what the biggest opportunities is to prioritize those and then right-size the learning opportunity that meets the moment. So for example, let's say you have— you said onboarding, so we'll start with that. That is an opportunity that you're going to onboard depending on your firm, 5, 10, 25, 30, whatever it is, number of people a year. You're gonna do that every year. It's something everybody that comes in the company goes through. So it's a high-impact moment to get somebody connected to your culture and awareness, and it's how they feel welcomed into the firm. And so you want to execute that really well. There's a huge ROI in nailing that. If you contrast that with somebody has a design presentation where they're introducing a new technology that can be used, maybe it's a new way of doing a structure like mass timber, or maybe it's a new kind of auditing framework, whatever the thing is, a new technology they can use for visualization. Maybe that one is like some people in the firm are going to use it. Sometimes it's kind of like almost more of an elective. And so you've kind of got this spectrum between really important strategic, this, this high usage, high volume thing all the way through something that'll get tapped sometimes. And so I think pipeline was meant to help you walk through and realize not all opportunities are the same. Some of them don't really matter because they're not going to impact enough people. It's just kind of a nice idea, but it's not enough to justify the investment. And to help you systematically think through what matters and what doesn't, to what degree, how do we build something compelling, who is our audience really, involving learners and helping shape the content and understanding if it's working. It's really meant to help people not get overwhelmed and have a more methodical way to move through it, if that makes sense.

That does. And one of the— one of those, I guess, the prioritization step, you introduced a subframework, an acronym for kind of testing the idea to see if it's something worth prioritizing. The acronym is DESIRE. Yeah, I'd like to talk about it. I also want to call out that you didn't use RICE, which is a acronym used in product, but like sometimes also comes into these conversations. So on a personal note, I was curious if you, if you played with using RICE and maybe you could explain what RICE is. And then I want to talk about desire.

Sure. I guess just take one more step back that I want to make another contrast to what traditional learning organizations are doing versus modern learning organizations. And I say this with love because I think this is all done from the goodness of people's hearts. But a lot of that, I said the fundamental unit is this hour-long thing. A lot of the ways people get to that hour-long thing and which one it's gonna be, is there some kind of professional development committee or the kind of people that run the lunch and learns or the forums and they look for volunteers. Like we have a slot, you know, coming up in April. Does anyone have anything cool they wanna share? And if Susan is available and has something she wants to talk about, that's what gets talked about. And then the same thing for May and the same thing for June. So we follow kind of an ad hoc discovery of what should be taught and are dependent on the enthusiasm and the whims and the schedules of who's available to teach. And I think that has served firms in a reason— it's not like it's been bad, but I think when firms figure out where they're trying to go, this kind of business challenge I talk about around, you've got all these emerging professionals, you want them to get to be a job captain or a project manager or a senior engineer faster than would've happened just by going through the old apprenticeship process, you have to become more intentional about picking and choosing which learning opportunities you invest in. And that's, I think, the, the shift, you know, that's starting to happen. So it's like, okay, well, how do you evaluate these opportunities in a somewhat rigorous, fair, non-biased way? I, you know, and that's where I think these frameworks. So you mentioned RICE, so it stands for Reach, Impact, Confidence, and then it's divided by Effort. That's spelling out the acronym. That's a, like, the standard product management framework. I definitely took some of the things, so I took the R and the I and the E. So the Reach is basically how many people will this affect? So we mentioned onboarding before. Onboarding touches everybody. Project management training doesn't touch everybody, but it touches a lot of people and AECs, NEC firms are constantly creating new project managers. That's the thing, you know, you're going to be doing 5 years from now, 10 years from now, 15 years from now. So reach is a thing. The I for impact helps you balance. Okay, so we know it's going to hit X number of people with reach, but does it really move the needle? Is this something that will teach them a foundational skill they're going to need to do their job better? Or is this kind of like nice to have? You know, it's like, oh yeah, it's kind of like, cool, I'm glad I know that. Almost like edutainment to some degree.

A play between reach and impact. And I think strategy ends up here too, is new employee onboarding where your reach— if you're only onboarding 5 employees this year, or only if you're onboarding 5 employees this year, low reach over time, high reach, but like in the immediate, low reach but huge impact, huge strategic alignment.

Yeah.

Teaching project managers to be better at it, not throwing them to the wolves. And to understand the way the firm does project management, big impact, right? And so you can go down the line of, yeah, so this thing will, if we do this and we do it well, it will move the needle. So what you're all, what you're getting at is, and the way that it's written is the reach times impact times strategy times enthusiasm times demand is the numerator, and you're gonna divide that by effort. So you're kind of, as you're going along, reach and impact, you're building a top line score, right? Yeah. Yeah. So strategy, That one's probably the most obvious. It's like, how well does this thing we're develop align with where the firm is headed? So for example, if a big strategic initiative for the firm is we need to improve quality, like, is this aligned with quality improvement? Cool. If we are trying to move into doing more planning work or consulting work on the front end of our projects, is this moving us towards building capacity to do more planning and strategy work? So that's a good variable. And you ask why I kind of moved away from RICE. RICE doesn't speak to strategy. So I felt like that was a super important thing that was missing. It doesn't speak to the next two either. It doesn't speak to enthusiasm. And I think to me, that's an important thing. It's like, who's going to teach this and how motivated and excited are they about the material? Are you just going to be like pulling them through the mud to have them do it? Are they going to be voluntold to teach this or are they going to be excited? So that's a variable. And the last one is demand. It's like, do people want to learn this? Now, sometimes people have to learn things they don't necessarily want to learn. There's some compliance courses we all go through. We don't wake up on Wednesday saying, you know what, I want to do cybersecurity training. But sometimes demand's not that relevant. But you can also say, look, we're talking to our junior people in the lunchroom, you know, or project managers are saying people really want to learn how to do XYZ skills. So like demand is a, it's a factor. So that's a kind of a balanced scorecard, if you will, of the kind of reach, impact, strategy, enthusiasm, and demand. And then the big thing you do is you divide that by effort. So how hard is this going to be for us to move the ball up the hill, like in order to make this thing?

I want to talk about the desire part and then talk more about the effort part.

Mm-hmm.

On demand and enthusiasm, I love having those two and I love them next to each other. Thinking about the modern learning organization and shifting from Susan wants to teach a course, or Susan's willing to teach a course about mass timber, to employees want to learn about mass timber, right? Or employees need to learn about mass timber. People are pulling us, asking us, saying, I want to level up, I, I want to do more high-performance design, and I can't do high-performance design if I don't know some basics. And I feel like that's— I don't know, I just, I like seeing those two because without that demand side, we might build something that is what people want to teach but not what people want to learn.

Yeah.

And I think one of the things I found myself saying over the last 3 to 6 months is I think that AC firms should be treating these learning experiences, these courses, whatever they are, more like products. One of the foundational errors that founders of product companies make, myself included, is building some stuff that you think is a really great idea, but you haven't validated a market for, or isn't necessarily strategic, but it sounds fun and it's right there and it's kind of easy and why don't we just do it? And that's fun, but it doesn't get you where you want to go in the long haul. And I think trying to bring some of that framework to thinking like, well, why? Yes, we have this course, like, and maybe it's even helping people think about, do we bring this into the modern era? So we have a recording from 6 years ago of a certain topic. It's maybe a little bit outdated. Maybe we want to modernize it, but do we even need this topic at all is a first question. So a framework like this helps you flesh that out. And if you've got 10 ideas, 15 ideas, 20 ideas as a learning and development team or whatever that committee might be running through this, a tool like this helps you just get some rigor behind how you're making decisions. But what's also interesting too is it surfaces disagreements. Let's just, let's move away from onboarding cuz that one's kind of more universal. But let's say that there's a course on, let's say that your firm has a healthcare practice. And you want to create a Healthcare 101. So a new emerging professional joins the company or joins a project team who's never worked on healthcare before. So they don't know the jargon, they don't know the medical equipment, they don't know the building layouts and the typology and the regulation. There's all this stuff to learn around healthcare. So the theory is we should create a, whatever it is, 30-minute, hour-long, 2-hour course to just give them the foundations of working in healthcare projects. And you might score this out and I might score this out You might say that this is like, your reach score might be very different than mine, or the impact score. I might think this will be transformational and you'll be think like, eh, they have to work on the project to actually learn these things. We can do this, but it's not really gonna move the needle. And I could say like, and it's gonna be super easy for us to do. And you might be like, this sounds like a big lift. We don't have any of this stuff written down. And so by having a framework like this, instead of you and I just debating our kind of internal tacit feelings on whether this is good or not. This gives us a way to kind of say like, wow, Susan, you put effort as a 5 and I put effort as a 2. What information do you have that I don't have? Or what information do I have that you don't have? You know, same thing for impact. Like we have theories of the case and it lets us really get a lot of that invisible thinking out on the table.

Yeah. Do you feel like, uh, that's maybe a weird question to ask, but do firms need to have some sort of audit to be able to move through this process, an audit of their learning experience. You know, I love a spreadsheet. A lot of our clients love a Miro board, and I'm seeing our clients build out some spreadsheets and Miro boards. Some are doing it more than others.

Yeah.

I feel like the firms that are doing a more thorough list of here's all the stuff we want to get our arms— or here's the stuff that— here's the learning that's happening. Even if they're doing it within a section, you know, maybe they're just doing it for all their Revit learning, but they're taking the time to write down the list.

Mm-hmm.

I feel like those firms are moving a little faster and the firms that take the time, they slow down to actually build an analytical, like, thought process.

Yeah.

Yeah. Yeah. So, and the question is like, do, do firms need to, is there another way?

And I don't know if there's an answer to that, but yeah, I mean, I think it's a little bit of go slow to go fast, but not only go fast, but go with confidence. So, you know, I think being very thoughtful and, you know, we think about this from product investments. We think about this where we go to speak and what events we invest in. We think about what topics we're going to cover and try and, you know, talk about from an educational marketing perspective. I think taking that little bit of time upfront to validate. It's good for alignment with the team also, because like, if you've got this kind of suspicion that my Healthcare 101 class is just a big waste of time and you don't ever talk about it and I just go build it and it turns out to be a big waste of time, that wasn't great. If it turns out that I was right and you were wrong, still, that's not great. I don't know. It just feels like there's a, there's an opportunity around, but then I will feel more confident Even if I thought I was right, you said, I don't think it's gonna work. And in the back of my mind, I'm like, what if Susan was right? Maybe, you know what I mean? And then I find I'm halfway through building and I realize, oh my God, she was right. We don't have any of this content. We have to start from scratch. I thought we had this all written down. It turns out we don't. And so it's just, I, I feel like it's worth a little bit of prevention here.

Another thing that I've seen is that by having the analytical plan, when you share it with someone else, you say, okay, I'd like you to record this lesson. I'd like you to record a 3-minute video on this topic. And they can see that it fits into this bigger story.

Correct.

That is well thought through, that will have an impact.

Yep.

They're much more likely to take the time to do it well, to prioritize it, because it doesn't— they know where it's gonna go. They have some confidence that there's a plan. Yeah.

I'm sorry to have put you in the role of skeptic in this hypothetical scenario, but if you don't mind, I'll go one step further. So let's say that we go through the exercise. And what we figure out is Susan was right, the effort was 5, but I was right that the impact is also 5. And so the numerator and denominator, it turns out it is gonna have a huge impact and we should do it, but it's also gonna be hard. That lets you feel like, okay, it's gonna be a bit of a lift, but I see why it's worth it, you know? And it also sets my expectations for we should do this, but it's gonna take a little bit because we actually have to build this content from scratch. And so it lets us move forward with a much more realistic idea of what we're signing up for. And it also encourages you to have good habits. The demand piece, I think, is let's go check with some people who have recently gone on to recent healthcare projects and kind of walk them through this and say, and try and validate before we build anything, would this have been useful? You know, or as we start thinking about who's going to teach it, maybe that helps us identify and like kind of bring them in early and be like, do you have the energy for this? Does this sound like something you want to own or Don't try. We'll support you as a learning development team, but like, do you want to drive this thing? So yeah, I find this, I don't know if we've gone on this for this too long, but I think this thing is a lot of where, let's put it this way, all of these things need to be thought through whether you do it upfront in a systematic way or not. And if you don't do it, it's gonna come back to you later. And it's better if it doesn't come back to you after you've already invested 20 hours or 50 hours or 100 hours, or you built something that nobody's using and you don't know why. Unless they've just been forced to do it and then they're like unhappily complying with it. Right.

You know, and your employee engagement score goes down.

Right.

What did we do here?

Yeah.

On the effort side, one of the questions that has come up a lot is we're used to the 1-hour format. Our teachers, our internal educators are used to the 1-hour format. We want to shift to short videos. We want to shift to breaking it down. We want to rerecord it. Or take this new thing that we're teaching and instead of doing it in the 1-hour form, we're gonna do it a new way. We're gonna do it in 5-minute videos or whatever.

Hmm.

Then this thing happens that we talked about a little bit ago of like, then things start to ratchet up. It's like, and we need to be in a studio and it needs to be perfect and we need to whatever, get the lights right and all the stuff right.

Yeah.

And that's not always necessary, that level of perfection to get the job done, to meet the user's need. You might, they might not care about that.

Agree.

Audio quality needs to be good, but maybe not, not everything, not for every course. But is there a floor for quality?

Yeah, yeah, yeah, yeah. I, I think that, I think about qual— not to say that I agree with you, by the way. I think audio quality really matters. There are $99 external mics you can buy that just raise the floor very high for audio very quickly. So I think that's a, that's an investment that's easy to make. I come at quality from a slightly different lens that we'll get to production quality, but I think of quality much more in terms of, is this thought through? Is someone just going off the cuff and just saying a bunch of things, or has this been considered? Has my time and life been considered as the learner?

Like you as the learner?

As me as a learner. Has somebody put care and consideration into putting something together that's kind of like coherent as a floor? But then is it concise? You know, is it like got a bunch of digressions that don't really add anything, or is it clear that one thing builds on another? Is it something I can use? First of all, period. But then can I use it in the next 2 weeks, 6 weeks? If I'm never gonna touch this thing, why now? Why me? You know? So I think that kind of quality in terms of did care and respect and thoughtfulness go into making this thing feels to me way more important than are we recording in 4K and have perfect boom mics and all the editing is clear because you could have something really well polished that isn't saying anything, you know, or is kind of wasting my time, but it's beautifully wasting my time. I would rather have something that's quick and dirty with a GoPro or an iPhone where it's very clear that this is a thoughtful, good learning experience that was designed to help me as a person. Do they feel it was designed or do they feel like someone's just banging it out? To me, that's, I think the floor is care.

Yeah.

I don't know if that makes sense.

No, it does. It does. And I also want to draw that there's a distinction between these learning experiences that you are going to assign employees to take and hold them accountable to take. They're going to have to take it. And I feel like to do that and have a good employee experience, you have to really care about their experience. If you're forcing people to walk a path and it's not a pretty path, they're not going to want to walk it again, you know, or take another learning journey with you.

And so you kind of get to a trade-off there, right, between—

and when I say pretty, I don't mean—

right.

I'm not talking about the sparkle of the video. I'm talking about like it was a path worth walking. Like I enjoyed the hike.

It was thought— it was thoughtful, right?

Yeah.

There's a couple of things in that. One, it's kind of balancing the efficiency for the expert to make the thing with the experience with the consumer slash the learner. Of consuming the things. And yeah, just kind of popping on a Zoom and talking for an hour may be the easiest thing for someone to put together. I don't know that that's actually true, but I think it's the comfortable thing, right? As you mentioned before, like this is what people know and it's what their expectation is. And you ask me to teach a thing on whatever, my mind's already thinking, okay, so 45 minutes of slides and 15 minutes of Q&A. We all know what that is. And you also asked, are we gonna go back to people that have done that before and ask them to remake it? And this is where, again, things like the desire score and having a framework are really useful. And if this is going to have a really low score and it's going to be kind of helpful for some people sometimes, I don't think I would take the subject matter expert's time to do that. Maybe the floor of quality is good enough. Maybe. If this is like Project Management 101 or Onboarding or Revit Basics or whatever these kind of foundational courses are, and learning experiences for your firm, I, you obviously the ROI that's built out through this financial model justifies putting more energy into it. The last piece of that too, and this is, I get into this some in this piece, and then the issue I just, we just published this morning around helping support subject matter experts in teaching is also surrounding them. You mentioned at the very beginning, you have a marketing team or you have skills in your company for storytelling, writing, interviewing. Packaging up videos, clipping. To the extent you can take a subject matter expert and just make it easy for them where they have to kind of just show up with their expertise, even to the extent like you've done your homework and you know what the important things are and you have a theory of a script of what to take them through. Anything you can do to make these very busy people enjoy the experience more and make it an easier lift for them, you can do. And so I feel like that's also part of this calculation is effort for whom? you know, effort for the expert, effort for the knowledge and learning team, for the marketing team, whoever's doing the work.

Yeah. Yeah. And I feel like there's what we're talking about, or like the language you're using suggests that it's gonna be a, you know, a short video. Let's say maybe 5 minutes, 15 minutes or something. It's gonna have a point. You know, someone's gonna be able to drop in, learn something like risk assessment for contracts, right? Something like that. I'm gonna watch a video, I'm gonna get it. I'm gonna be able to come back to this in 3 months when I actually have to do a contract. But I feel like there's also another kind of learning that can happen, which is a conversation between an expert and an expert talking about a policy that's maybe somewhere on the intranet that is less focused but rounds out someone's understanding of why something is important.

Sure.

So I just want to call out that there's multiple modalities. Oh yeah. And when we think about the employee's experience, the learner's experience, there's room to play. It's not like everything has to be super linear in every lesson for it to be a good experience.

Yeah, like, I mean, the example that's coming to mind, and people are going to learn about what Christina Williams has been up to at Lionakis, you know, they'll hear about it through the podcast and webinars in K Connect this year, but she's taken a longstanding program, which is their Design Technology Bootcamp, which was all live synchronous instruction. And over a number of, I think it's 52, 54 hours a year, for new people who are gonna be design technology specialists and be embedded in project teams to support the teams with Revit and all the other tools they're using. And so what Christina's been doing is, well, let's try and do some hybrid formats. How about the instruction-y part of this where I teach you what design options are or how to do Keynotes or how to do whatever the kind of the technique is. I can, instead of that taking me 30 to 35 minutes to teach you, if I'm efficient in making it prerecorded, maybe that's 18 minutes. Maybe that's 20 minutes, and I can cut out some of the sidebar stuff and just simplify it down to the core things you need to know. What they're going to do is watch that together as a group and then discuss it. Or, you know, we've seen— you and I were in some beta cohort calls yesterday where it's like, I want you to read our AI policy, right? I want you to watch this short video about how we're approaching AI tools, and then I want you, through a link in here, to book a 15-minute meeting in my schedule to talk about what you learned, what your ideas are, how you think AI might help you. And so these like multimodal hybrid environment, like that's what I think a modern learning or experience looks like. It's not one size fits all. But the side benefit of that is that in the Lionakis example, sometimes people learn better if they can pause and they can go back, you know, or they can break it up into like Maybe it's a, maybe it's 18 minutes, but I just want to pause this, think, go get some coffee, come back, watch the next 9 minutes, make some notes, open up Revit, maybe try the thing, keep watching. And letting people go at their own pace is more accessible for more people, but it also means that I can go through the course and then 6 weeks later, 6 months later, and I don't remember how design options work, I can come back and rewatch it, right? It wasn't contingent on me being live with the instructor in the room. So it's also thinking about recall and retrieval. And then now that it's digitized, anybody at Lionakis who's trying to figure out design options, they just search, you know, on their internet and AI search is going to pull them exactly the answers they need out of that course. So you've done a lot of things by digitizing this, both for the immediate experience of someone going through that learning program, but then also, you know, downstream and for others.

One of the things that actually came out of some of our work with Lionakis, preparing for them sharing their story, is thinking about that retrieval while you're building the course. And there was a specific example with Autodesk Construction Cloud, mm-hmm, slash Forma, where they were breaking out the courses and they were like, one of the things you're gonna have to do is add, you're creating your project in Forma, you have to do all these things, and then you have to add your users to the project. So instead of making the add users and project part of a broader video, They pulled that out as a separate lesson and separate video because they're like, that's a thing people are going to have to come back to a bunch of times probably. We need to make that recall really good. So they optimized for the recall of a specific step that they thought people would need to recall.

Yeah, it's almost like if you're producing Saturday Night Live or you're producing The Daily Show, you're thinking about it in terms of clips and segments. And those clips and segments are going to be standalone videos on YouTube downstream. If you're thinking about your course that way, it's like these are key moments that will stand alone. But added up, they make a continuous experience. And I think again, that canonical hour-long thing, maybe, maybe it's an hour long because it wasn't as efficient as it could have been. So maybe it's down to 45 minutes, but 45 minutes can still be the right length, but maybe it should be 9 5-minute segments or 5 9-minute segments or 3 15-minute segments for exactly the reason that you're saying. In addition, it makes it easier to maintain that knowledge too. So if something changes that was in minute 37 of your hour-long thing, but you've gotten it broken down into a 9-minute segment, maybe you just have to replace that 9-minute thing, you know? And so you've got less of a, a technical learning debt that you're taking on.

Yeah.

Yeah. So this is, this is it, right? We're talking about modern learning or this is how you're thinking about this, right? You're thinking long-term strategic value, working back, thinking about the needs of users, thinking about retrieval, while you're thinking about the person that's gonna take it, maybe they're gonna take it live, maybe they're gonna take it on demand. How does AI factor into letting people easily extract the piece they need? So you're thinking on multiple levels, which is, I think, getting back to the stress that you talked about at the very beginning. It's like, this is a huge opportunity. I don't wanna screw it up.

Yeah. Yeah.

So then it's also worth some more strategy upfront is kind of what we're talking about.

I love it. Yeah. So in that Lionakis story, they rethought how they're going to lay out the course because they realize how it's going to be consumed. They realize it's going to be consumed just in time via AI search because someone is looking for information about adding people to the ACC/forma plan.

Right.

And I feel like there's another thing that can happen there, which is understanding that your course can be consumed by someone who has a problem and is just trying to solve it might change the courses you develop, might change the people who are interested in developing courses.

Yeah, there's a lot there. Yeah. So it's like both— I mean, here's the thing, beginning with the end in mind, right? So the end in mind is this might get consumed by AI Search. The end in mind might be people who aren't necessarily the target audience for this, but may have useful knowledge in here that helps them continue solving problems on their project. The end in mind might be the person who goes through the course but doesn't remember what it was 6 months from now. And in the old way, at the Linoss example, they had all the recordings of all of those sessions, but they were, you know, as we talked about before, the long hour, hour and a half, 2-hour version, and people just didn't go back to them because they were very linear assets and they're hard to break apart and hard to find exactly what it is you're looking for. And so for Christina and her team, it's like, it's a little bit more of a lift in 2026 to redo this content and do it in this way versus just show up again in 2026, teach it using the slides we already have.

You mean in 2027?

Well, I guess they redid it in 2026, so like they could have come into 2026 being like, look, we've already got this stuff developed, let's just do it. Or they're like, nope, we're gonna start over because we believe In '27, '28, '29, this'll save us a lot of time. And also once this stuff hits Leo, that's the name of their Synthesis intranet, that means that knowledge is searchable for everybody else in the company. And so it's immediately put to work as an asset. So that return on knowledge is much higher, which justifies that short-term investment, which I would argue is probably gonna save them time over the long haul. And Christina from Linakis has also been working not just on the design technology stuff, but she was telling me about some of the practice bootcamps they're putting together and working with a subject matter expert who, once she told them his job was to show up, talk about his expertise for 10 to 15 minutes, and then go away, like, he's like, great, that's all I have to do. That's all you have to do. We're gonna handle the editing, we're gonna handle the packaging, we'll create a course out of it, we'll run it by you as a draft. And so in that case, you're increasing the return for the organization. You're lowering the investment for the subject matter expert and maybe you're increasing the investment a little bit for the org, but you still come out net positive when you think about what capturing this knowledge can do for the organization. So I think it's that calculus. I don't know of anybody that's actually— I take it back. Jess Purcell and the team at Shepley Boffins have actually calculated out the hours of doing it the old way versus doing it the new way. In her KConnect 2025 talk, she goes through that in detail. Fascinating stuff. But most firms haven't calculated, but they can feel it. They're like, yes, this is gut test. This is going to make us more efficient and more capable, and it's a good use of our investment.

I gotta say, yeah, that I think that this might also play into how we think about resourcing knowledge and learning in our organizations, right? Some of our clients, a minority, have a knowledge manager. Or a L&D department. The majority do not have those roles, but we are seeing those roles on the rise. We're seeing a combined knowledge and learning manager role on the rise, some trends there. And I feel like this ROI calculation for how much the organization can actually get out of a training if it's done really well and if it's done in a more modern way.

Yep.

And that to get there, someone needs to do the work, so we might need to give someone time to do the work. I still think that there's a place for knowledge and learning to be embedded in other departments, but I, I think the calculus on all of that is changing a bit.

Oh, 100%. I mean, when we started Knowledge Architecture, the first KA Connect we ran, my goal was to get together the named knowledge managers in the AEC industry for a roundtable in Chicago of 15 to 20 people, because that's how many I think there were. And instead, I mean, the history of KConnect is we got 80 people. They weren't all knowledge managers, blah, blah, blah. It turned out to be a whole thing. And those knowledge managers were at SOM and Gensler and HKS and Arup, large 1,000-plus person firms in general. That's where they were. Now we're seeing knowledge managers at 50 to 99 person firms, not infrequently. It's, it's becoming a thing. And that sea change, I think really in the last 10 years, we started seeing more and more knowledge managers at the under 500 person firm. In the last year, we're seeing it at the under 100 person firm. And I think we're going to start seeing it now at the under 50 person firm. It just seems to be that firms are running. I think you're exactly right. They're running that ROI calculation. And part of that ROI calculation is we can take so much load off our experts, not only just for the knowledge capture and the creation, although that's true, but we can give our experts more leverage. You know, it's like if I'm thinking about just, we're just name dropping experts here. If I'm thinking about Corey Squire at BORA as the director of sustainability, look at what he's done over the last few years there in terms of building out their knowledge base, putting his book on their intranet. Again, working with some other experts. It's not just Corey on an island. He's working with a team, but that was a good part of his job to build that out. And now they're in the learning management and they're gonna do the same thing all over again. And the whole idea is on one hand, the expert gets more leverage cuz they can scale their expertise and they can spend more time on researching, innovating, working with clients, mentoring employees versus answering the same 101 and 201 level questions all over again. And the user experience for an employee, it's like, I don't have to wait until Corey's not busy. Corey might be on a job site, Corey might be on vacation, or Amy or any of the other kind of experts at their firm. I can keep going in the flow of work. I'm not disrupting my expert. And there's this whole flow state that gets unlocked. And I do think that, yes, a platform like Synthesis is a key part of that. But you're exactly right. I think that whatever that knowledge and learning management team is, whether those are FTEs, whether that's Christina, we had brought up multiple times, she's 50% DT, 50% knowledge management. Katie Robinson at LS3P is the Chief Marketing and Knowledge Officer. She's got a split role. Somebody, I think, needs dedicated time in the week where that's the thing they wake up in the morning and that's what they're thinking about. Not, it's the 9th thing on their to-do list they hope to get through by the end of the week. That just doesn't feel responsible. You're not gonna become a modern learning organization doing it ad hoc and under-resourced like that. You're just not, you're not gonna be able to take advantage of these new opportunities in this new technology. And so, yeah, I think that's part of this leveling up thing. This is a platform shift and it requires rethinking people, process, and culture as well as technology.

Yeah.

I don't know, you were the first to report back to me another change where I think it's fair to say more architecture firms than engineering firms have had dedicated knowledge and learning resources. But I feel like I'm seeing this change too. We just had an 80-person engineering firm last week say they're hiring a knowledge manager. I know of another one who hasn't published the job description, same size, doing the same thing. And this feels like that's changing too.

Yeah. I mean, you know, I'm an engineer and I don't want to like pigeonhole engineers, but the math is pretty simple.

Mm-hmm.

You know, and I think our engineering firms, particularly the small ones where they can allocate, you know, they're maybe more limited in their resources. But they can be more nimble in making adjustments to their teams and structures. Yeah, they're like, yeah, let's, let's— I mean, frankly, give it a try.

Mm-hmm.

Right. Let's really give it our best effort. Give a modern knowledge and learning organization our best effort. And that means resourcing it. And we know that all the engineers here are billable and on projects. We know that our marketing team is already running lean.

Yeah, right.

We know that HR is already running lean. So where is it going to happen? A person, right?

Where is it going to happen?

Yeah. And in some cases that might look like— this is an engineering organization, but another one of our clients, they had a couple planners whose projects stalled, like funding, I don't know, something like that. So we're like, great, it looks like we have about a month of runway. These couple people are gonna be underutilized, let's make that time really intentional and give them something meaty to work on.

But if we're talking about that same organization, I think we're talking about the same one. The COO is running point on this operation and she's, she's the one that made that call because she's deep in building the modern learning organization for them. And so she's pulling in, right?

She's the person who wakes up every day and is thinking about it and, and saw the opportunity and connected the dots and then made it happen. But if no one is playing that role of I'm at the table, I'm at the leadership table every day thinking about how do I resource this particular aspect of our growth, it's going to be hard to identify the opportunity.

Yeah, this is all long, medium to long game stuff. This is I'm working on the important but not urgent. I'm working on the business, not in the business. There's like 9 ways of saying the same concept. I'm sharpening the saw. I can keep going. There's so many clichés around this, but yes, it's that. Right? And I was on the phone with a client this week who isn't on the modern learning organization journey but wants to get there. And I'm gonna pull back to AI for a minute. One thing I have seen is firms sometimes either being too far on the AI is fantastic, it's gonna do everything for us, or they're afraid that it's not gonna work. There's ways to over-index on being too bullish or too bearish on AI. I, I feel like they were more on the, we just need to point AI at our 30 years of experience on our server and then we can ask it any questions that we want. And I certainly haven't seen that, you know, really technology exist. I don't think it's there. And part of the reason I don't think it's possible is that data is super messy. There's all these final underscore final, no really this time it's final stuff. And a person that hadn't worked on that project wouldn't know with the right answer words, let alone an AI. And that's a person with context in that firm. And so there's some, I think, kind of magical thinking that we just apply AI to the problem and it's gonna solve all this for us. And I think that what I've seen is the firms that are making the most progress don't think that's true. Yes, it's true, AI will help us do some things, but we're gonna need to go build that digital knowledge foundation and learning foundation that's gonna allow our people and AI to understand how we work, what we've done, what our approach is. That's the way that it's going to work.

I think they're really treating AI like a new employee.

Yeah, right.

If you think about it that way, it's like, wow, they're going to— we're hiring someone and maybe it's— maybe AI is actually 3 people or 4 people, or maybe it's 1 person. I don't know what the FTE balance is. And they're seeing— they're seeing the ROI that way. You don't go hire someone unless you think they're going to give you some ROI.

Mm-hmm.

And then they're training the asset, the intelligence, mm-hmm, with that same level of attention that you would onboard a new employee to, to make them fully productive.

Capturing knowledge is how you train an AI like Synthesis.

Capturing and teaching.

Writing down, right?

Yeah. Like you don't ask a new employee to go read your 10 to 20 terabytes of project files and then do their job.

Right.

Which could be any job. Right. You figure out what job you've hired them for. You give them the— you tell them to read the section of—

Yeah.

Section of a terabyte that actually pertains to them being able to do their job effectively. And then you measure them on their ability to do their job effectively. General intelligence, sort of like AI intelligence, is obviously a little broader than that. It can do multiple jobs, not just one, but same kind of principles. And I feel like that's— yeah, I feel like, again, It's math.

Yeah. So I think another way of saying that is the modern learning organization is thinking about the learning and development of its employees as well as the AI, right? As well as the agentic layer that's gonna be part of your business. We will, I don't know, I can't say this with 1000% confidence, but this is the worst AI and the least AI we will ever have in our businesses today. And so if you believe that's true and we'll have more AI and it will be better, Making an investment so that the AI understands your projects, your people, your standards, your processes, your culture, your values, just like you would want new employees to understand the same things. You gotta capture it and put it in a format that is accessible to both humans and, and agents.

And bringing this back to the, the modern learning organization pipeline, like the pipeline we've talked about, developing good courses is just one little part of that. Like it does help train the AI, but it is it's this whole flywheel that is getting going, right? Where AI is the, the ability to get that content when you need it, when the learner needs it, is driving the creation of new content. It's driving the resourcing of new content. It's driving our willingness, our firm's willingness to invest in developing that content because you're not just getting it to the 5 new employees that joined this year. You are giving it to all your employees and you're giving it to the AI.

That's right. Yes, that's exactly it. And I think it's this jump. We— this is step 7 in the model. This is getting a lot of people's attention. Once they understand that this is the end game, I think it's changing that, that ROI calculation. But the part that's not here today, but if you squint your eyes and you think about where this is going, it's not hard to imagine that even a learning and development expert experience, if you do the work now and you capture the knowledge and you build the foundation, and build the infrastructure and the processes to support it. It's not hard to imagine a world where we have AI tutors in the future, for example, in platform. So it's not just like Susan wants to learn to be a project manager today. It's like, yes, you will go through our Project Management 101. You'll be supported by an agent and by AI search that can answer your questions about project management. But it's, it's still not exactly like Susan really wants to build a skill around contract negotiation or having difficult conversations in an AI that could tackle all of that knowledge plus understand Susan and be able to mix and match resources and learning experiences to help you on your learning journey. That is, feels like it's not in the too distant future, but again, the step to put yourself in position to be able to take advantage of that future when it comes is around knowledge and learning management, which is back to your, do we need better resources and teams to support us? To be able to get toward that destination. Yeah, it's exciting. I don't know. Yeah, it's fun.

Yeah, I love it. Back to strategy. We're gonna get some spreadsheets out of this podcast.

Sure.

I wanna talk about end users and when we involve them in the design process of a course, of a learning experience, of redesigning or rethinking about how we do learning in an organization. If I remember correctly, in the pipeline, learners come in for the first time named, it's like, you should talk to them in the middle around right-sizing and design-build. But in the earlier steps of prioritizing and identifying, they certainly could show up. But there's also a little bit of a risk, or I've seen this risk play out, where if you bring people in too early, well, there can be some information asymmetry.

Sure.

Between what the learner knows about the big picture learning opportunity for the organization and what the champion or learning team, right, committee knows about what they want to accomplish. And things can potentially get derailed if you throw too much at the learners, ask them too many questions before having something kind of material for them to see and understand, like why you're even talking to them about it.

Yeah.

So I'm, yeah, I'm curious about your thoughts on when it's like the right time to bring in learners is to make sure that we're incorporating actual learner feedback into the design without overwhelming them with this question.

Yeah. So I have, I do have them in step 1. There's an informal conversations with learners piece in there. So that's really around building the list of what are the opportunities we should be thinking about. And that is meant to counterbalance. And I say this with, with, I mean this in the best possible way. All of us were emerging professionals at one point in our career. And based on how we remember, I think we think of what would be useful, what they might want to learn, right? Or from our position now as more senior people, we're like, I need them to know this. They may not know they need to know it, but I need them to know it, right? And those things are both true, but it's also true that things have changed since you were an emerging professional. And so they have ideas, maybe there's new technology or they come from another firm and the other firm was using some new practice or new process that would be a really good thing to use at your firm. And so I think definitely engaging them as part of your identification process is really, really good. I think you're right about the— so, so there is information asymmetry, but it can also cut both ways to some degree. You don't always know what your audience needs, and so it's good The more time you spend talking to your audience, the better. I think there's another area where it's really important, and especially now I've been hearing things like everyone's attention spans are super short. Everything we make needs to be 90 seconds or 2 minutes because people are used to TikTok. And I think there's some obvious evidence that those tools are popular in the culture. But there's other, there's counter evidence that people watch the Acquired podcast and Joe Rogan Pod— those are 4-hour-long episodes, right? People used to watch movies. Now they're streaming entire television shows. So if something is interesting enough to them, they will keep going, whether it's educational entertainment or whatever it is. And so I, I'm not, I'm very skeptical of this argument that people don't have the attention span. I do think it's true that people have lost some of their endurance. I read a piece last week that it was in Cal Newport's newsletter saying that at the NYU Film School, right, super prestigious film school, they're saying that the film students are having a hard time making it through full-length films because they just get antsy, right? And they're like, I need to check in. I'm out of the conversation. I don't know what's happening. And I don't, I don't doubt that's true. I've definitely paused a show and been like, What else is it? How the Warriors doing? You know, whatever it is, I get, I kind of like, I don't dispute that premise, but I think if your thing is valuable enough, people will finish it, which is again, another benefit of on-demand and letting people pause and go at their own pace. It's another argument for instead of an hour-long thing that you're presenting somebody, if it's a 9-minute thing, maybe that's more aligned. I can do a 9-minute thing, take a break, do another 9-minute thing. Maybe that's more aligned there. But I think talking to end users and seeing if that's true. Is useful, really. How are you consuming content? If we were to design this and you're saying we're thinking of 3 different ways, you don't have to do what they're saying, but you can certainly get their feedback. It seems like a useful thing to do. So yeah, I think you should involve them to the extent possible. I think involving them all the way along. In fact, in the issue I just published today, I kind of linked to a clip of Todd Henderson from Boulder Associates One of the things they did is they actually had the learners become the teachers. So they had a healthcare expert named Kate who had built all these medical planning tools. And the strategy Boulder used was, we're gonna have these emerging professionals interview Kate, become the expert on the tool to the extent that they're gonna present to the office, they're gonna document and present to the office how the tool works. So there's also an opportunity to offload some of that burden from the expert, both from teaching the thing, but then also of being the go-to person always for these questions. And so I think there are a wildly interesting number of ways to involve learners. That's just the, that's just starting, getting started on that.

Yeah.

Yeah. I love that. So in the pipeline, there's a pilot step, which is the place where like, yes, you are going to involve learners. But if you, if that is the first time that you are involving learners, then you're probably going to have to rebuild a bunch of stuff because, right, you will likely spend a lot of time and energy and money to build the thing and cross your fingers.

You hope they like it. You know.

Yeah, yeah. When we're launching intranets, there's a phase that we generally do 3 weeks before launch called early adopters, where you bring in some people who've just never touched the intranet. They're, they're not content editors. They aren't probably not on the leadership team because they kind of know what's happening. You're just trying to find people who don't know what's going on, give them the intranet and have them give you feedback. And information always comes out of those that is very helpful to the team. Small things the team can adjust.

Yep.

And then go live 2 to 3 weeks later. For the learning experience, I think it's a very different model. You still need these first readers, people who are actual learners, representative of the broader learning body, not someone else on the Internet Core team or the learning team, but actual learners. And what I think is going to happen in pilots, or what we're starting to see happen, is that you're going to get information back that you're going to need more than a couple weeks maybe to go back to the course and adjust to get it ready for prime time. The other interesting thing that I'm noticing with the pilot phase is most of our clients are launching like one or two courses or maybe one learning path at a time. So you do have the opportunity to pilot something, make it a little better, launch it. Give it to all employees, learn from that, and then you're back in prioritization and right-sizing for some other program, some other learning path. And you're able to take all that learning from the first one you rolled out and roll it back in to make the next one better. So there's a bunch of internal feedback loops that are happening, and I think organizations are going to be running content through the pipeline kind of like one content area at a time as as stuff comes out of prioritization, you know, it starts moving through. So you might be in a place where you're actively distributing your Bluebeam training. Yeah, because that one felt high impact and relatively easy, good desire store, and someone was passionate about doing it and it was a good test course. So you're like, you're actively distributing that, you're piloting maybe some like Revit training And then you're in prioritization and right-sizing for PM training. Yeah, like, I don't— where do I even start with PM training? That's going to take some time to even whittle down.

Yeah. And do I, do I do PM training as a monolithic block or do I start with something important like negotiation or difficult conversations or fee, whatever the thing is? Maybe I break it down and I go after the most important part of project management that we're struggling with, that people want help on. Back to your research again. It's like, yeah, there's probably components of project management that are more important than others for you to invest in right now.

Yeah. So I think that everything bagel approach to the pipeline is, is actually really good. It's like get things out to users piloting as soon as you can so that you can start to get feedback that you feed into the other stuff that you're developing.

I think that's right. I mean, this is going back to me thinking, I think people should take a product management lens to learning and development. The goal when we go into beta for a product, we're gonna put knowledge agents into beta in Q2. I'm not trying to figure out in a beta if doing knowledge agents was a good idea or not. If I don't know that by the time we get into beta, I haven't done my job as a product manager. If I don't understand what the core problems to solve are, the core jobs to be done, the use cases that knowledge agents, at least the initial ones that they will unlock, and have a high degree of certainty that those matter and will make an impact and we get to beta, I haven't done my job as a product manager and I can keep going down the line. So what I'm hoping happens in beta is as much as you try, you always have the curse of knowledge as a designer of a product, of a learning experience, that there are things that don't make sense that you can't see how they don't make sense because they make sense in your head, you know? And so some of the things we're seeing is clients are piloting these courses is, I didn't understand what this question meant in a quiz. So therefore I, I couldn't make sense of the answers because I don't understand actually what you're asking. That's a clarification. That's a, that's a bug, right? That, that, that somebody's finding, or they get a couple lessons in and there's something missing. It's like, I don't understand how we went from topic 3 to topic 4. It feels like it, this was mentioned in topic 4, but never explained. And it feels like something was missing. And so there's maybe a gap in something that needs to be filled in. So it's that level of cleaning up that I think that we— that's what we see in product. Someone's using it in a way we didn't see coming or something we thought was super clear in a button and they don't understand what it means.

Yeah. One of the ones I heard this past week was we mentioned this kind of learning modality of like you watch the video, now talk to— call the expert internally, schedule a time with the expert internally.

Yeah.

The user got to that is part of the pilot and they're like, do you really want me to schedule time with you?

Right.

Right. Like it was just a different way of working that we were throwing at them and there wasn't enough context for yes, this is expected, this is welcomed. Maybe they need to put an agenda in there, something to get the user more comfortable.

Right. If we're talking about the same example, I think it also wasn't clear how. Do I hit you up on Teams? Do I just come to your desk? And so in other examples, people have put a link in for like a booking link. And it's like, yeah, book a 15-minute meeting with me. This is what we're going to talk about. Yeah, exactly. It's creating more clarity. That's exactly the kind of thing that a good pilot—

And you're just not going to be able to see that yourself.

You won't be able to see it.

Of course, you put all the clarity in you can.

Yeah, yeah, exactly.

Whatever you thought.

Book a meeting with me. How was that not clear? Yeah, totally.

Right, right. Yeah. A lot of organizations— I mean, every AEC organization is doing some learning, whether it's named, whether it's traditional, whether it's modern. Learning is happening. Otherwise, the organization probably doesn't exist. And some of those processes have a lot of momentum. I'm thinking about new employee onboarding or project management onboarding. It's like, yeah, maybe it's not defined, but people are becoming project managers.

Yeah.

Someone's telling them how to do it. So there's, there is some momentum around it and within the pipeline, I feel like they're kind of, we can think of those learning experiences at the end of the pipeline. Those are rolled out, people are engaging with them, they are happening. And so one way to approach them is in the, the end of the pipeline, there's an activity around measuring activity, measuring experience, measuring outcomes. And then using information about what is actually happening to go back to the beginning and think about your inventory and your prioritization. But I think there's a lot of baggage. And politics basically tied up in opening that box, that kind of Pandora's box, let's just say PM training and getting everybody who's involved in that, who are all a lot of powerful people in the organization to actually maybe face that it's not working or that there's differences in how it's being done. Yeah. So I feel like there's a lot to be explored in just trying to measure some of these existing processes, but they're hard to measure because there's— they're not— they might not be formalized. There's no feedback form, right?

Yeah.

Right. So how do we go about capturing some information about whether they're— how they're going and whether they're working right? Whether— yeah. So we can prioritize them.

I think that's all. That's all good. And I mean, picking your battles is very important. One of the ways that I think it becomes— I think why I like— why I kind of rewrote RICE and made it Desire is I think it gives you a few more elements to think about helping to pick your battles. So let's take this. We have a project management course. It's heavily in-person. It's 3 straight days. Everyone knows that people are burnt out by the end of it. They don't retain a lot of it because it's hard to go to class for 3 days and just take a bunch of— and then they don't apply. A bunch of it, you know, immediately cuz they're in CA, so they're not applying stuff that's at the earlier stages of project. I can keep going on and on, but it's like, okay, but like you run it through a desire score and it's fine. It's like a middling score. It's not great, but it's fine. And it's like, so is this the hill we wanna die on this month or this quarter? Or is there this other healthcare 101 thing that we figure out that like, man, we have nothing. There. We're doing nothing. And if we made a change, the gap in where we are today and where we'd be if we did this is so much higher. That just makes more sense to be the next thing. And it could be with also the project management when you do the desire score, maybe one of the reasons the score is where it is, is the enthusiasm for changing it is very low from the people that need to do it. It could be, or it could be that they're like, we agree, this is awful, please help. You know, and, and maybe it's all the things. And so I think, I feel like that's why it's good to have a depersonalized, depoliticized way to try and break down the conversation and have people kind of weigh in on, yes, everything could be better all the time. It's true with products, it's true with learning experiences. And so really figuring out what the things are that are gonna move the needle the most for the organization is what— that's, so that's part of my answer to your project management thing. Then you get to, okay, no, we've identified, we're aligned, project, the project management training. We need to revamp it. I think that's when you can start doing things like, it's true, maybe we don't have feedback forms, but you can go interview the last 5, 10, 15 project managers who went through it. What of that stuff stuck with you? What of this stuff have you never used? How would you help us design this if you were going back into the— it could be that, it could be somebody who's whatever the step in your organization before, maybe it's a project architect, maybe it's job, like however you get to project manager. Involving them, it's kind of like by the time you get to that role, you know, some of the things around project management. So what are the things you think gaps you need to close or seem the most scary or daunting for you? And I just feel like you just get into design then, right? And that's sometimes you don't have perfect data in design, so you have to go find what you can.

Yeah. Yeah, I love that. That seems like maybe a nice place to end this is like continuing to loop back to the user. What do they need to learn? What are they seeking? And let that inform the whole process.

I think you and I are aligned that like doing some strategy research work upfront and prioritization work upfront is a really good idea. That can go too far too. You could spend months and months and months chasing your tail, figuring out the perfect sequence and the perfect strategy on what we need to go build. Meanwhile, you may have a huge knowledge gap in your organization that could be filled. Maybe it's your onboarding, maybe it's some new technology, whatever. Maybe there's a company initiative around AI or safety that could really use support of a learning and development experience. And you can kind of skip the line or like do enough. Yeah, we're all agree this has a pretty high score. Let's just go build something. Let's right-size it, let's run it through a pilot, but let's not wait until we have the plan figured out of everything we're gonna build in what order for the next 5 years. This isn't— as much as this looks linear and all models are broken, you know, but they're useful. This is a looping process and you're going to continually— it's almost like a fractal, right? You'll be kind of like in a pipeline as an organization, but then every course will go through different phases and should go through different phases and kind of like getting started, but thoughtfully. It's that balance of don't rush into it, but don't take too long doing analysis. And I hopefully that there's like a balanced way to do this.

Yeah. Well, this was great. Thank you so much for—

Yeah, it was fun taking the time to walk through it.

It's a fun way to explore the framework.

I'd encourage people as they're kind of implementing and using this and they've got questions or refinements, like this is the first version. We're going to continuously be learning and unlearning in terms of modern learning organizations in the pipeline. Like, let Susan or me or the, the team at Knowledge Architecture know. We'd love to hear it.

Yeah. Awesome. Thank you, Chris.

All right, you got it. Thank you. Hey, thanks for listening. If you like what you heard, you can subscribe for new episodes on YouTube, Spotify, Apple Podcasts, really anywhere you get your podcasts. And if you've got comments, feedback, or ideas for future shows or guests, you can email us at smarter@knowledge-architecture.com. Thanks for listening, and we'll catch you next time.

Pittsburgh, House Street, MacArthur. I wanna creep! My favorite transportation, no matter what your station. Wherever I need to be, take the ART bus. Please, to the bus! Cuz I don't need to drive a car, I get some money on a plastic card. Anywhere you want to cross the bay, just make sure you don't stay too late. Oh my, what Dublin to Millbrae, Orinda, San Bruno, or Ashby to Concord. Our Delhi City, my favourite transportation. No matter what you station, wherever I need to be, I take the BRT. Which is the best? Which is the best? Which is the best?