Learning at the Speed of Change: Adapting L&D to Your New AEC Workforce | Kristina Williams

I'm very honored to be here on the KA Connect stage and happy to share our experience with all of you today. A little bit of context for you. So we're Lionakis. We were that photo that you saw in Chris's presentation from— we've been around for a while, even before that photo. Actually, our founder was the first state architect in California in response to the Great Earthquake. We've got 240-ish employees across 6 offices, all in California. We serve in the core markets of education, healthcare, and civic, and we do architecture, structural engineering, interior design, and so much more. I've been with Lionakis for 14 years. I started as a Revit specialist, and as Chris mentioned, I had been a Director of Design Technology for many years, and just up until very recently, 2 months ago, I went into a full-time knowledge management role, so I'm very excited about that. Still have yet to see what this all means. But today I'm going to share with you why we felt compelled and crazy enough to create over 230 and counting short format videos from scratch, despite having hours and hours and hours of past live recordings for those same topics. So most of them had many versions of live recordings for the same topics. So some background on why we did that. Much of that content was created for an annual training program that we hold, our Design Technology Bootcamp training. This is something we've held annually for over 12 years, and it's a leadership-selected opportunity for a small group of individuals. Those individuals are representative of each market, service, and office that we have. We tend to have somewhere between like 3 and up to 10, but not often do we have that many. We cover intermediate to advanced topics like model management and design technology stuff, visualization, coordination, production, and standards. And these individuals are in studios. They're, uh, they're not on track to be in design technology. Mm-hmm. They're designers, architects, structural BIM technicians, and when they go through the boot camp program and they graduate, they're all the while still embedded in their studios, but they have the opportunity to become a Design Technology Support Specialist, or DTSS is what we call it. So, it's an added role for them. And these individuals, they help with tech support, peer-to-peer support on their projects. They help test and advocate for technology that we're implementing in DT. And by doing this, we can grow our collective intelligence around design technology across the firm. We can scale support without scaling headcount. We can put DT expertise closer to the work because they're working on projects, whereas us in DT, we're overhead completely. And then we can start to recognize emerging leaders from this Because they're seen as go-to people in their studios, we started boot camp in 2013 with just a handful of people in Sacramento, and we've been holding it nearly every year since, always growing and adapting how we deliver it to today, where we have 30 DTSS across all offices, and we're hoping to add nine more from this year's cohort. And all of these individuals we spend. spent on the low end around 50 hours of training with each of them. And that doesn't even count all the people that have been through it and are no longer at Lyonelkes. So all of this is really just to say that we've spent a lot of time not just delivering this program, but thinking about how we're delivering it and if we're being effective when we're doing so. So just a little bit of a glimpse of how things have adapted over the years, the things we've tried, things that have worked and haven't. Our very first boot camp was a lot of hours. I still look back and I think it's incredible that we even got buy-in for this. It was full days of training, which nowadays seems pretty uncommon, but back then was pretty common, especially for Revit training. Those full days were sandwiched between full days of project work, and everybody traveled to Sacramento to do this in person. We felt like with the curriculum then, Maybe the outcome really wasn't the best outcome we were going for. I think we were trying to dump everything we knew into their heads and then put them out in the studios and expect that we've just got more DT members out in the studios, which isn't realistic. And I think that because of that, we dialed in on the curriculum a little bit, and it was hard to get buy-in on that program. We also felt like The full days in person were really great. I kind of miss those days because they were really immersive and we did— we bonded a lot. But also the mix of project work and training. We thought maybe we can just focus all in one month, all in on training, and we could have better retention of the topics that way. So we tried that. We reduced the curriculum a little bit more. With technology training, I started to feel like there was a lot of mimicry going on. Meaning, like, we had them follow along hands-on in the course, but we're doing something on the screen and then they're following, and we're doing something on the screen and we're following. And I had that experience myself along the way when I was learning Dynamo, and it wasn't sticking because I was doing the same thing. I was just— I was at a conference, I was watching, doing, watching, doing, and it just wasn't sticking. And it wasn't until I struggled through it on my own, doing something fully immersed, that I started to learn it. And so that's where it occurred to us that we needed to introduce that struggle for them to be able to do it on their own, iterate, think about what they did, and then go back to, um, more learning. And so that's where we introduced, uh, these exercises. Um, the exercises are just, uh, little handouts that they follow that they do individually or in small groups, which gets them to do the struggle or the, um, desirable difficulty is a much better way of saying it. And this we've been doing ever since. And I think it was a step in the right direction for sure. Then of course the pandemic came and we were forced to do everything virtually, which was a blessing in disguise because though it was really nice to be in person, we were taking advantage of everybody's time and their presence traveling. So we had to cram everything into those full days. Now we can do everything virtually hybrid and you all know the drill 'cause we're all doing the same thing. But it allowed us to spread out the sessions and shorten the sessions. And we've been actually doing a similar schedule ever since with about 50-ish hours of training, 1 to 2 hour sessions. And then Synthesis LMS came about and Chris had asked me, how might bootcamp look different now that we have tools like Synthesis LMS and technology like AI available to us? And You know, maybe we would've gotten to this pondering eventually, but that question really just like opened up our eyes to a new vision. And we went back to the drawing board and completely revamped Boot Camp for this year's Boot Camp. So this was going on in like late last year and early this year that we were planning all of this. And enter a new era of Boot Camp. And at the time we were really hopeful of our approach here. And now that we've done it, we're really happy with how it's turned out. And this is a new era, not just for bootcamp, but for a lot of different learning opportunities at our firm. And so going back to the title of this talk, the Learning at the Speed of Change, I want to address what we're reacting to with the change. And Chris has addressed a lot of it here too, that with social media, you kind of wonder, are our attention spans shortening? I don't know, because we're all able to watch long movies and listen to long podcasts, things that are relevant and interesting to us, we give our full attention, right? So again, Chris had put it really nicely at one point, a new perspective that maybe we're not, our attention spans aren't shortening, but that we have a lower tolerance for information that isn't useful to us at that time, right? And especially when you're learning something at work, yes, it's useful for your job, but it's, you know, you gotta find when it's relevant. And with AI especially too, that, It is able to distill a lot of information into one answer for us right when we're looking for it. So in a way, the learner expectation is changing for how we want to receive information. And then of course, the doing more with less, the more codes, more documentation we have to do, more litigation. We're all subject to this pressure of doing so much. I heard recently that it's to deliver a $50 million project, you're now expected expected to do so in half the time than 10 years ago. So we're all being crammed into this. We have a lot less time to squeeze in that important information along the way. And so here's what we set out to do and have accomplished so far. Like I said, we recorded these short format videos, and these were intended for a future audience. So again, back to those live recordings, they're Unpleasant to watch. They're hard to go back to. No one ever really wants to. So those were not meant for a future audience. Those were meant for the people attending at that time. And even then, was it really great for the people at that time? I don't know. Uh, so we recorded a series of, uh, videos and then of course included our exercises as well. And this is a course on our learning center, and you'll see more about that in a bit. And with those short videos, we had a choice here. We knew that if we sent them off and assigned them these short videos, and then we wanted to reconvene to do some exercises together, what's the likelihood that half of them would show up and have not watched the courses or didn't have time? Some of them forgot, some of them feel guilty, whatever, right? No matter what, we're starting the session, getting people up to speed, getting people on the same page, and we're— we've now lost And where we could have maximized exercise time. So we decided to do a live watch party to just watch them all together. And yeah, it's weird because I'm the instructor, I'm the one that recorded this, and I'm right here in the room and I'm just playing it for you. At first, I was worried that it would seem like lazy in some sort of way that I'm like, I didn't feel like coming and talking to you. So like, I don't know, this, it was weird at first, but then I got really comfortable with it. And it's been a pleasant experience. So we watch them together and then we know right then and there that we're all ready to start on the same page with the exercises at that point. And they know that they can access them on demand later because they're recorded. And when we set out to do these recordings, we aim to say it short, say it clear, be as concise as possible. When I was watching the past live recordings, I would even watching myself, I would think, man, I really have a tendency to tell them a lot of just-in-case things. And I knew that about myself in the back of my head, that I'm thinking, I don't want them to think I didn't warn them about this thing, so I'm going to tell them all the just-in-case scenarios. And that's who I was as an instructor. And I think that really added to the noise that they were hearing and prevented them from retaining the truly important information. So watching those and then recording these helped me be a better instructor, and I've been teaching these Same topics for like 10 years. Revit hasn't changed all that much. And also, when watching the recordings, I felt like, man, this is really slow. Like, how many of you watch your YouTube videos on 1.5 speed? I mean, I'm pretty sure we're all at that level, right? And I figured, I think we can process information faster than we can say it. And some quick searches led me to believe that that is true, that we can process a lot faster than we can talk. And it's generally safe to increase your videos to 1.5 speed. So we are conservative and we do in our editing, uh, 1.25, 1.2, sometimes higher speed so that when you're playing our videos on normal, you're hearing them at 1.2-ish speed. And so we just wanted to, by default, save people time. Uh, and, and then they can slow it down if they'd like to also. When we created the recordings, we also had AI search retrieval in mind. And so in design technology, we're used to when we're creating tutorial pages, we know, okay, I want, I want to be able to send this URL to somebody when a tech support question comes up. So same thing with these videos. I want to be able to send a URL of a video and then have also have AI search retrieval pull that same video up. And we could do the same with like longer videos and then chapters that it could pull up. But what I like about this is also you can update the videos a lot easier if you need to change something. You just swap out certain portions of those videos rather than a full course with chapters. So this is an example of a Revit course of linking in a DWG file, and it's more about if somebody may have experienced linking in drawings in Revit a bunch of times, but they forget the little settings in that dialog box. So now there's a video that we can just send them, or it can bring up in the search retrieval too. So we went to town on creating all of these videos, and then speaking of quality of production, we did not aim for high quality production. We just turned our cameras on, talked, and just went for it. And part of that was keeping in mind the sustainability of this method for other instructors. So everything we were testing, even the recording software, we wanted it to be really simple for anybody to use, knowing that we wanted to extend this format across the firm. And so what our general plan here is where we would just turn on our camera for a few seconds in the beginning, and then we would turn it off after those few seconds. And we're— we edit it so we're in the lower corner, so you're not big on the screen. I think right then and there, if you're asking someone else to record a video and they know they're huge on a screen, it's really intimidating for them. And do you know that once you turn off your camera a few seconds, that you kind of are a little bit more at ease when you're doing the rest of the video? So that was our goal, and the quality for us was in the clarity and conciseness of our messaging. Those videos and exercises came together as a course, and this is our course catalog on the Learning Center's Synthesis LMS. And then a collection of courses comes together as a learning path. And this year we held 2 boot camps entirely in this process. So the The live watch party, hands-on exercises, and then availability on the Learning Center afterwards. So our regular DT bootcamp program, about 50-ish hours of training, and then new this year to us as a firm was a visualization bootcamp training. And I have been a little out of touch myself in my role, even as Director of Design Technology, with the visualization process, right? We have a visualization team, and I was just thinking how— You know, we have a new— we had a new design media specialist on board that recorded these, and I thought if I needed to get up to speed on this, this would be my favorite way. I would actually look forward to, to watching these, and I'm so glad that we capture the knowledge from this individual in this format. And then speaking of instructor burnout, right, like I mentioned, I've been teaching these classes for 10+ years, and even though I could teach them in my sleep every year, I would still have to prepare my dataset files and make sure everything's working properly, refresh my memory. And we would spend a lot of time preparing for these courses. And I had always wished that I had more time to focus on the more enhanced parts of it, which are the exercises and assessments of what's working and what's not. But I would find I wouldn't end up having time to do that because I had to prep so much. And so that led to a lot of burnout and burnout in teaching the same topic over and over again. We're hoping that These videos have a decent shelf life so that we can start to realize the benefits over years of not having to prepare for those at all and then focus on the next-level tasks. And so what we found was, I guess, interesting, but we should have expected this, is that we would deliver the message in the live watch party, and then when they set out to do their exercise, they would be watching the videos again right away. And I thought, wow, that actually was my experience in the live training in past years where I would say something, then they'd start their exercise and they would ask me questions about what I just said. And I would always think, was it me? Did I say something? Like, how do I tell this in a better way? And it's just the way that we all learn. We need repetition and we need it to sink in, and then we need to do it and go through that cycle. So it was actually really cool to see them just They would put their headphones on and just start the videos and do their exercise. And they went into the exercise feeling a lot more confident that they could achieve it themselves and way less burnout from us cuz we weren't repeating ourselves right away. And now they know because they're in the habit of going to the learning center that once it comes up on their project, they can go right back to it again. So that was really cool to see. And we really enjoyed this hybrid version because We could use video for what it's best for, the clear, concise messaging that is easily rewatched, and then live interaction for what it's best for, which is the engagement between humans and talking through and working through the challenges of the exercises. So some people, I think at first, will think that learning through video takes away from the human experience, and I'm now starting to see that it actually enhances the human experience. Because it takes away that repetitive robotic talk or just the fundamentals that we're kind of repeating over and over again and allows us to spend our energy talking to each other and problem solving. And that really is where, as an instructor, I would feel the most energy is working through those problems. And so when we were talking with our president about what we were doing in DT, she was really supportive. of that and opened the doors to carrying this format into other areas of the business. And I truly believe that executive sponsorship, as it always is, is just so critical in this situation. So, you know, getting into the practice side, we're addressing different changes and different challenges. We've got the hybrid work environments and the dispersed teams. The project leaders or the mentors are subject to the doing more with less stress. And so, as Chris said, they're having a harder time finding time to do the mentorship, even if they really want to. And that the apprenticeship model, it's not, it's not dead, it's eroding a little bit. We want to keep it alive, we just need to adapt it. We also have growth on our mind. And when we're acquiring a new firm, how do we onboard an entire office more efficiently? And how are we making sure that they're not getting the disparate voices? Is it Brent and Jennifer's way of doing things? How do we clear that up into, uh, one message? And so when we were going through our firm calendar of learning events for the year, we thought, you know, where can we apply this new format? And it turns out, pretty sure we can apply it in all sorts of areas, even on the practice side, even though some of the tacit knowledge in those areas is a little bit harder to turn into explicit knowledge. And so we sat in a room with a lot of our project leaders and started with pain points. And in that room, it was more about like our gut check of pain points. Since then, I've heard from other firms using RFIs and items that come up in quality checks and using those tangible items as your pain points to determine where do you want to focus your time on for your learning, right? And if you're gonna invest time in creating these recordings, let's make it where it's gonna really count. And so we came up with some topics for our pain points. And then most importantly here is what was feasible. So on the design technology side, that was easier for us to create a large volume of content because we've been doing it a lot, right? Not as feasible on the practice side for all the reasons that we know. So we just started small. Let's just do 4 courses, 4 topics, one each quarter, and then we could do 4 on the business side for project management workshops as well. They're different teams, so let's go for 4. And I think it's important to start small here. And we did that same thing. We held a live watch party, did hands-on exercises. The watch party was a little bit different here because it was a big Zoom call. with everybody on the practice side together during that live watch party versus the smaller cohort in bootcamp. And the first one that we held was a cartooning course, and this was recorded by one of our busiest and most experienced architects. It was a very engaging training session. When people left it, they were messaging us afterwards with how awesome that was and How they had never thought of cartooning that way. And it was just nice to see people like jazzed after a training instead of drained. And we, you know, held it the same way. We had the videos, we went out to breakout rooms to do exercises, we went back to the main room to watch more videos. And in the exercises in the breakout rooms, we had had those that were less experienced kind of sharing their screen and driving.

Mm-hmm.

So that we could still have that struggle and not have the more experienced people just chiming in and answering it. Because what we, we did want the more experienced people to attend the course because we want to get to that same message, not the disparate voices. And by attending the course, they're seeing that message that was documented and that that's the Lionachus way of doing things. And that's how I should be directing my team. And in fact, now this is like packaged up nicely and I can just send them there. So it's important for everybody to know what's there and to attend the course for that. And then in doing so, the person that's creating the recording is responsible for understanding the Lionachus way and for getting perspective of their peers in different markets to make sure that when we're documenting it and codifying it, it is, you know, truly our current best known practices. So getting on the same page, we can then hopefully grow together from there. And then similarly on the practice side, like I said, I know it's harder to take that tacit knowledge and turn it into explicit knowledge, but I do think that we can pull down a lot of those pieces and make and turn it into these videos so that we can cover the foundation so that the energy of that mentor is better spent on the project specifics and the more challenging parts of that topic. So we were really sensitive to the instructor experience at this point because we know that our busiest practitioners, this is going to be difficult. We want to extract the knowledge, but we don't want to drain them in the process. That architect that did the cartooning course, this was definitely a new process for him. A little apprehensive at first. It was unfamiliar. But when he talks about it in hindsight, he says it was way better for him to create a few short videos than to prepare for a 45 to 60 minute presentation. And he's actually a kind of a good spokesperson for us, for others that we're trying to get to do the same process. We also offer part of a knowledge coordinator's role to facilitate this experience for the practitioners. If you're looking to do this, I don't think that this needs to be a full-time person. This isn't this person's full-time job. And it, again, it depends on how many you're tackling, right? Like with the 8 courses this year, this was very manageable, uh, as a part-time role. But, uh, using AI, they would help take a bullet point list or a brain dump for, from the practitioner and put it into our PowerPoint template and start to develop the slides. I was even thinking just in the last session that maybe we could have them, you know, when they're brainstorming, some of them we had 2 practitioners recording for the same class. So when they're brainstorming their outline, we could record that and put the transcript in to then create the outline of some sort, right? To just like, let's use AI to help get their content out. They don't need to sit there and spend time creating slides. And so the knowledge coordinator also gets them comfortable with the recording and what to do there. They'll edit the recordings, upload the courses, and then facilitate the live training. And I think that was one of the most rewarding parts for our instructors, was to show up on the day of the training and just answer questions. And that— I really loved being able to see that for them. And then these are nothing special, but just where— what we're trying to do next is just come up with some templates or things that can get them thinking about their recordings and how to outline them, to prompt them with questions. One of the things I found has been the most challenging is to get them out of the mindset of the 1-hour presentation. So a lot of times I'll, if I'm working with some, some of them in the beginning, they'll think like, okay, so I'm going to take my 60 minutes. I'm going to turn it into 10 5-minute videos plus some time for Q&A, right? So we essentially just have the same thing, a really long course that someone would have to watch, and it's just broken up by videos. So what I found is allowing them to just brain dump their whole story will help, and then try to pull out the pieces of what do you really need in this training session? And then to start with the exercises, what do we want people to struggle through and to— what do we want to get them to do themselves? And then what videos do we need to do to support that? And if you wanna have more videos on the learning center for context later, sure, we can create those. And that way when somebody comes across 'em in the flow of work, then they can see more content at that time. But let's focus on what we're really trying to do in this live training. So this was just like our specifications training. They're trying to recreate their Specs 101 in those short videos. A little bonus that came out of this was that the course was so nicely packaged up that Anybody could take it on the road. And it happened with the cartooning course because there was a whole office that wasn't able to attend that course. And then there was one of our engineers that happened to be going to that office the next week. And he's had enough experience on cartooning and even with architectural sets working with our teams. So he just went down there and delivered the same exact course, facilitated the exercises, answered The questions, and he didn't have to prepare a single thing. And it was the same single concise message about how we do cartooning. So about that live training, there was definitely a period of time in this process where I was like, why are we even doing live training at all? And I'm talking about the kind that I've been talking about so far, right? Not the leadership training and the things around culture and trust. But just the broadcasted, here's a topic that you're probably not going to come across in the next few months, maybe some of you are, that kind of live training. And it made me think about that debate I always hear about between universities and firms in our industry where, you know, we're questioning why— how come when somebody graduates they don't know how to put a set together? And the university is saying that Our job is to teach them how to think critically and to expose them to these topics, but it's really on the job that they're going to learn how to do that. So I figured, you know, there are a lot more outcomes to training, to live training, than just skill competency. And so here's what I think some of those outcomes are. One is it's the firm saying that this is an important topic. This— we have a way of doing this. We have a LionAUKUS way of doing this, and we want you to know about it. We reinforce that message by repetition, so not just saying it once but carrying it through to other trainings that we're doing and through studio meetings. Also, just again, the doing the single concise messaging, the trying to get away from the disparate voices, that live training can help with that by having everybody attend and see what content's available. It's filling in gaps, taking people from what they don't know they don't know to what they know they don't know. So once they know they don't know it, then they can— they're more likely to go seek it out, that they know there's a way, right? It's giving them a little bit of a roadmap. So we have this document that helps guide people to what's important for the LionAucus Way across all the stages of a project. And I envision that we'll be adding some of our Learning Center content to this too. And I really see the Learning Center and the live training as contributing to that broad knowledge and giving that roadmap of what's important. And then also the Learning Center serving the ability to fulfill the deep learning as well, so that when somebody comes across it on their project, they can go on that offshoot of diving deeper into courses, watching them again, And then that paired with mentorship and time and experience can start to get to that skill competency that we're looking for. And then bootcamp has its own purpose for live training. It's the— if we had just had any, all the Revit users just come across the content in their flow of work, then we will have never had somebody that has been through and has seen the big picture from start to finish. that has really connected with us in DT and has formed an appreciation for best practices. There's the power of the cohort and having that cross-market collaboration and camaraderie with the members in their boot camp year, sense of belonging with DT, the emerging leaders. There's so many reasons to still hold the live training on a cohort method. So just going back to live training is really, to me, just the In this format and those topics, it's just to plant a seed of knowledge, to provide awareness, to touch on the skill competency, but not to actually achieve it depending on the topic, right? And then our learner experience, I touched on how people were kind of ranting and raving about the format of that cartooning course. And we did, we had somebody message us saying that they feel so grateful to work at a place that has so many thoughtfully put together resources. And this gentleman over here, I I don't know if this gets at your question that you had earlier about like the apathy for if I don't have the right resources, you know, how do you feel about that in your job? And so I do think it matters to show that you care about the learner's experience too. And there's the learning center. So accessible content is the Linocchus way, but it's easy to navigate like YouTube. And then just what's next for us here? We hope to We wanted to take some of our onboarding content and put it into the learning center in the same format. Also, reboarding of existing employees is everything that I've been talking about so far, just getting everybody on the same page. And so when we created these courses, we had not only search retrieval and updating of those in mind, but we had the idea that we didn't want to repeat courses multiple times or multiple videos across several courses depending on the audience. So we broke up our courses into these 3 topics: big picture, core, and deep dive. Big picture is the what and the why. Here's our Lineaus way of doing something just generally and why we do it. And then core is how to do something from point— get from point A to point B in as short of steps as possible in the clearest way. So that's for somebody who it's their like first or second time doing this. And then the deep dive is where we can just dump everything else. It's just the just-in-case stuff that I always wanted to say, the nuance, the project-specific scenarios. We can put everything there so that when somebody's experiencing this in the flow of work and they're actually seeking that knowledge, then they can arrive at a ton of it if they need it. And then so a new hire would just see the big picture stuff. They don't need to know how to do it yet. They just need to know that it's there when they need it. And then, like, a bootcamper would watch both the big picture, because we still need them to know the why, and then also the core too. And so, I had the pleasure of delivering our first new employee orientation, just the DT portion. So, what we do, and probably a lot of you do something similar, is a new hire will meet with a lot of our department leads. and just get an orientation to that department. Okay. But, and then in the meantime, they're meeting with their market leads and their supervisors to get to work on their projects. So I had been used to doing the DT orientation for every single new hire that came through the firm. So talk about bootcamp courses that I've been doing for 10 years. Well, I don't know how many new hires I have said the same exact thing to for 10 years. And so I really wanted to package that But nowadays, because I've participated in so many of these recordings, anytime I have to show somebody something, I immediately want to record it first and then show them because I just know it'll be worth it later. So I went, recorded these, and then I had 3 new hires that started last week, and we had our regularly scheduled 1-hour DT orientation. And what I did was I broke that up into 15 minutes of meet and greet. And this is where my favorite part is, where we just get to know each other and talk to each other and I understand their background. And then I had them go back to their desks and watch the videos. Because again, if we were to assign them a bunch of videos along all of the onboarding, that just seems like an overwhelming task, right? But here, let's do it again in this hour that we already had scheduled. And then when you're done, because I know exactly how long it's going to take, come back and we'll do Q&A. Okay. And then what was— what they were really grateful for was they knew it was available to them on demand on the learning center, because how many people are drinking from a fire hose when they first start and they don't retain half of the information that we tell them, which is totally understandable. So now they can go back, you know, once they need to, they can go back to the learning center. My favorite part about it was that the meet and greet, I guess this never needed to occur the first 15 minutes of a meeting, but it, because the videos make it so accessible, I was like, this doesn't need to be in a conference room. I turned it into a walk and talk where I was lucky that we were all in the same office. So we went on a walk around the block. This was a picture from our walk and it was just so refreshing to just, you know, get the human interaction was actually enhanced. Because I had the videos that I could just send them to afterwards. I could get more work done. I wasn't drained. I think a lot of times I come to the onboarding meetings just a little bit— I try my best because I'm so excited to meet them, but I'm a little bit drained knowing that I have to say the same thing again. So this, I was so excited for this meeting, and it was great. I can't wait to have all the others that are doing this follow a similar process. Another thing that we want to get into is Chris had mentioned knowledge agents, and I really feel like knowledge agents can supplement the learning experience by being able to continue beyond the course. So the course can get you that foundational information, and then you can dive into discussing with an agent or assistant how you can make it apply it to your project-specific content. So this was just a little one that we did, but in that cartooning course, the first video was your drawing organization and how to break out a building into multiple areas. And so here with the sheet numbering and naming assistant, you can let it know your project details and it works with our standards for sheet numbering and naming. It'll ask you how many stories, levels, buildings, and start to work with you on your actual project task beyond the course. And then ultimate goals would be to have ubiquity around this so that everybody's saying, oh, there's something on the learning center for that. Go check that out first and then we'll talk later. And then you know that when you're sending someone there, that it's an experience that they can trust and that they would enjoy when they're doing that. You're not sending them to a 1-hour recording just to get 5 minutes of information. And to just build that culture of knowledge seeking. is what we want so that it's natural for people to not only know that they should, but to want to seek out that information. And then my last thought here to share with you is just that no matter who you are, if you're— you don't have to be a learning and development expert, though I wish we had one on our team. I think we would have rapidly gone through this whole process. No matter who you are, if you're developing training or you're delivering training, I think if you consider yourself a designer of learning experiences, you'll really start to put your shoes into the people that are learning from you or from others. And that just the way that our designers are considering the user experience for the built environments that we're designing, we can consider our own people and their learning experience too. Okay, thank you.

Thank you, Christina.

Yeah, thank you.

Awesome. Okay, so, all right, you have a new job. You're no longer the Director of Design Technology, but there's like all these videos with you on them.

Yes.

So are you going to—

I thought about that even for the onboarding. I'm like, I actually have to explain in the onboarding that I have a video saying you're hearing from me and this is who I am, but you really, you're gonna be working with this new person because that's where we are right now. And I'd rather just get into it because I'm gonna be saying that same thing to them anyway, right?

Yep.

And yeah, with the boot camp content, it was a little bit of a bummer because I'm like, I actually will not reap the benefit of this next year. But I think of it as a gift that I gave and that I'm still around. So when they're hearing that voice, they'll be like, oh, Oh, Christina knows Revit. Like, you know, like, with that, or like, you have all those videos.

Do you think that your team will re-record them all next year, or you'll leave some of them with you and they'll just do—

Leave them all. Yeah, we talked about that in the beginning, even thinking, you know, what if this is somebody that isn't here with us, uh, in a few years? And I think back to Design Technology members who are no longer here with us, and if I would have minded if one of the videos was from them. And no, I don't think I would if the message is still what we want and it's clear and concise, then does it matter? And we could, you know, I would imagine that we would just preface it with like, hey, this was, you know, so-and-so from, uh, they were a part of the DT team at one point and they were really great and had a lot of knowledge to share. You know, like, it's okay that people aren't here, you know, anymore.

How did you, like, um, in making the short videos, making them clear and concise, did you Script them? Did you edit them heavily? Did you do multiple takes?

So we, well, part of it was because we've been doing these topics for so long, but we did have outlines from before of what we went through in the full like hour to 2-hour course. And so we would just look at those and try to piece like, what would a new hire need to know about this, right? Like for instance, we have a production standards document. Okay, the new hire just needed— needs to know that we have this, generally what's it— what it's about. And then, um, I think, yeah, we would just look at the steps and figure out what could fit into a recording. We just kind of rearranged the outline. And I think if I were starting from scratch and I didn't have that outline, I would do something similarly to what I was doing with the practitioners, which was just, you know, put the whole story out there and then think of the— what you want them to do. First.

Start with the exercise and work backwards.

Start with the exercise.

How did they like that approach of starting with an exercise and working backwards?

They like it. It definitely, something clicks with them when I say that, but then they're like, wait, how do I do an exercise? What exercise? What's a, can we just do a quiz? And I'm like, I think really you want them to struggle through something. So let's think of something fun that they can do. And we can use AI for that too. You know, what are some fun exercises? What are some fun, games or something. And so they like that, especially when, when we can offer help with the exercises. And then once you tell them the exercises or bounce some ideas off of them, they get really excited because they're like, wow, people are going to love doing that. And it's actually going to get them to start thinking about the things we need them to think about rather than just what we're telling them to think about.

And then they can work backwards. So this is then the content I need to create to set them up to be able to do the exercise.

Plus usually you need more than just that. Like you're not just teaching to the exercises in the videos, but you want enough context to, to, to do that. Yeah.

How that busy architect that you talked about that did the cartooning that went first, um, he ended up thinking it was a good experience. How did you kind of, how'd you convince him to be first?

Um, so this is where that executive sponsorship comes into play because, and I'm glad you asked this question cuz I wanted to elaborate on that. Because in boot camp, in our last session a couple weeks ago, I— we usually rely on some of our past DTSS to record some of the topics, and I had a hard time with that, honestly, because it's just coming from us in DT. They're really busy. There's not that executive sponsor or leader in the room saying that they have to do it. So that was rough. And I realized how important it was because Brian, the one that did the cartooning, this came from Laura, our president. We were, he was in that room determining the pain points.

Mm-hmm.

In that room deciding we were gonna do 4, and we were all talking about, okay, you're gonna be the first one. And it's just, we were all committed at that point. And he felt, I think, more accountability to the whole firm that way. And it was a lot easier from our perspective to, sure, to get, it was more of us helping him through than like, Getting him to do it really. So that executive sponsorship is really, it's really key. I can't, it's just, can't stress it enough.

When you were watching yourself for the first time, did you like kind of wanna do the, like, but you know, this other thing might come up. Like, did you wanna do some of those just in case digressions or just feel like, no, I know I've got this.

Yes. Like, and I edited them out or I noticed, I noticed I was about to do them. And then the way I do my editing and I tell other people is I'll just I'll just go through and if I mess up a sentence, I just pause and then keep going. So you don't feel like you have to redo the whole thing every time you mess up. And then you just edit that stuff out later and, um, you could do the transcript editing. So you're just deleting words and that's really fast. Uh, Camtasia has a nice audio aid for that. Um, but yeah, I would find myself doing that and then I would just stop and go back to what I was saying and, and not do that. Or, and then edit it out. Yeah.

So in making the video and then watching it with the other folks in the room, were you, did you have that same instinct or were you kind of like, did you understand that like, we don't have time for this now, Christina?

Uh, I did have that same instinct. I said, I, I think a lot of times we would end the videos and I would wanna tell the just in case. And it, I did many times. Um, but it, you get used to that and yeah. And it's okay cuz it's not in the content for later. Uh, So yeah, but, and part of that is the exercise where you are getting into some of the nuance, right? Um, so yeah. Yeah.

Yeah. That's cool.

Yeah.

Um, we have questions and then we see up here, they're all on this side of the room for some reason. It's like this, this little quadrant.

Hopefully. Oh, test. Hopefully a quick and easy question. You're in all the videos or someone's in all them. Did you consider using any of the AI training creation things where you do a script and it talks for you? We did consider that and we talked about that a lot in our LMS beta cohort on how many people were attempting that or not. We didn't, yeah, I think we just wanted to try it ourselves and we weren't sure how we felt about it and we knew we could do it. So we haven't dabbled in that yet. I do feel like there is that, you know, if we're concerned about taking away from the human interaction, I feel like that does that a little bit more. It is more efficient. So I'm not sure how we stand, where we stand on it yet, but we haven't tried it. Yeah.

Hi.

Hi.

Um, regarding those, uh, just-in-case topics, is that something that you would put in a more deep dive video?

Definitely. Yeah. And, um, I thought we would do a lot more deep dive in bootcamp, but we ended up not. We kind of focused on the core and our plan is to record more deep dives later and actually to more crowd crowdsource the deep dives because I think the deep dives are a lot easier to capture because like I can go to any of my DTSS and say, here's the core and the big picture that we recorded. Is there anything you would want to add to that? Because you're doing this on projects or you have a more market-specific thing and they can just be like, oh sure, I'll just work. And I'll tell them it can be a minute. I don't, it doesn't even matter. Just, just brain dump. And there's no like, You don't have to intro and you don't have to explain all the fundamentals. And so that's kind of our plan next is to start curating some of the deeper dives.

What's an example of like where the core would drop off and like what a deep dive would pick up?

Um, let's see. I'm trying to think of like, I don't know, I think of design options and I think it's so technical in Revit. People are probably like, but design options in Revit are something that, you know, you show them how to set up. different options. And, uh, even though a lot of Revit users should know that, they often don't. So we find that we have to cover and make sure that they know to do that and not to just make a bunch of copies of their stuff. So we're getting on that same page in bootcamp, but then design options can be pretty challenging and complex, and people don't want to use them because of that. So we actually have a past recording. It is short, from a DTSS that's not here anymore, where he just was able to work design options, like, really well and in a way to do the entire building in a design.

Almost like a little case study.

Like a little— it was like a little case study. Yeah, case studies is where I feel like a lot of the deep dive will be. And so, so that, even though he's not here anymore and it wasn't a prerecorded course, because it's short and it's very concise and it's just him flying through how he uses design options, I feel like that could— that serves well as a deep dive as it is right now. Like, yeah, we might re-record that, but no one, no one does design options the way he does. So I don't know if I could record it or recreate it that way because he was so skilled at it. So I, I see a lot more flexibility in the deep dive because that's where people are looking for that information. And so they're— they may be like, okay, I'll take anything I can get. I'm gonna just sift through everything that you're putting here.

Would those ever be assigned or would that just be, I discover it via search, you notice it, I get to them?

I thought they'd be assigned to at least the bootcampers, but now I think they're just more flow of work stuff.

Yep. Yep.

Yeah. Cool. Where are we?

We're at Jess and then Andy, I think, or I don't know. I lost track of that other box.

So I feel like one of the things that People are hesitant about moving onboarding to video is losing that in-person connection. So I love the like sandwiching either side of it. Have you gotten feedback yet on how people feel about that, like in the middle videos? Yes. I asked them specifically. I told them that they're the first people that we've ever done this with and how excited I was about it. And I told them why we're doing this so that they're like filled in on the whole process. And I think they appreciated just understanding all of it. And so I think for a while I will just explain and over— almost over-explain like their experience here so that they know that this isn't just us sending them to go watch videos. This is something because we care about their experience and we care that they may wanna rewatch this later. Um, so I think explain. And then I did ask them for feedback. I'm like, I wanna know exactly how this is.

Mm-hmm.

You can say you hated it, just Tell me. And they, they, they loved it. They, they were like, this all was great. And they loved going on a walk with me. And it was— it felt great. Uh, the, the one thing I noticed there, and I was going to mention it on the speed slide, is that we had someone that was an English as a second language speaker. And the— I think the speed is a bit of an issue there, which is totally understandable. I can't imagine watching an Italian video at 1.25 speed, right? If I, um, so I've asked them, uh, to see if the slower, like if they slow it down, if it's, uh, if it's better for them. And, um, we'll get feedback there. And I think I'll, I'll really rethink the speed if that becomes a, an issue. So yeah, we got feedback and, um, when we did the Q&A, I was, I think I want to enhance that part a little bit more. It's like I want to do an assessment to make sure that they understood it, but I don't want to feel like I'm quizzing them at the same time. Because I— it did feel like a little bit like, that was great. I have no— not really many questions, maybe like one little general question. And so you can't really tell how much they got out of the videos, but they, they all seemed really— because they knew they were part of this process. They seem pretty invested in it with me. So yeah.

So in the same way that you kind of pioneered the hybrid learning, then the practice took it, do you think other people might take your onboarding approach?

I sure hope so. That's what I'm going to work on next is I know they're going to love, especially the ones that are repeating it like I am, being able to record and not have to say, say that again and to be able to, you know, put more energy into that human interaction side of things. I think they'll really want to.

Yeah.

If anybody doesn't really want to, then I'll just dig into why and we'll figure out like how to make it better. Yeah.

Good.

Yeah.

So, Todd.

Yep.

With testing—

there we go—

with the lot of focus on videos, curious what proportion of your content that you're developing is video versus, you know, written down and what made me question it was the question about all of the stuff you're self-editing out of your videos, all the little details and corner cases and oddities feel like a great bullet list on a page. Is that what you're doing or is it all video?

So it's all video currently. And I'm glad you asked that because we have had on our mind, what do we do with all of the written tutorial pages that we have? Because currently they're off in the tutorial page land, and this is on the Learning Center, and how do people know which one is more relevant? So we, we want to maybe transport over those to underneath the videos. And then as far as the proportion, it is all— I think I mentioned that it is all videos and lessons currently in this new format.

Right.

But I want to consolidate our written stuff. 'Cause we do have that written. And then as far as the just-in-case stuff that I'm editing out, um, I don't end up getting too far into it to where I could actually take it and then use it. You know, I stop myself and then, and don't do that. Or I, I've gotten better at not doing it at all. Um, and those are the ones that we plan to do next. Like maybe that's what happens next year for bootcamp is the DT team will record the deep dives.

Mm-hmm.

And, and just show them, you know, what's available. Or there are some topics we do, like even sometimes design options, maybe half of them already know it. So I feel bad going over the core stuff, and maybe we can do some sort of mix and matching based on where their level is at that time. And so half of them are looking through deep dive stuff and half of them are going through the core content. So I think that answers your question a little bit. Okay. All the way in the back. Okay. Yep. We're actually really curious about how you're assessing the knowledge that people are gaining from these training sessions. And, you know, we're— I love the idea of having these bootcamp sessions, getting people up to a certain level, and then being able to have them transition into forms of like more integrated DT leadership. But how are you actually assessing the widespread knowledge that people are gaining from these training sessions and then, um, being able to leverage that. So, so that has been on my mind this entire time through bootcamp, and it felt like we were always spending so much time preparing and we had all this other stuff to do that assessments were all— they always fell by the wayside, even though they shouldn't. And then this year we spent so much time recording that we again couldn't do the assessment. So that's what I was mentioning in this the next level stuff is what we can focus on in upcoming years, and assessments are one of them. And I really want to dig into that because currently we don't have a method really of verifying the content. Yeah.

Yeah. I think we have time for one more.

Sure. I'll take it.

So curious on sort of you do the live sessions and then there's the on-demand. Are you beginning to measure Anyone who's going back and seeing the content, or are you reaffirming later on, like, hey, go back to this?

That's another thing I want to get into, and I'm excited too. I mean, we just delivered this, right? Like, we finished the courses in February or March and then delivered bootcamp all the way up through last couple weeks. So that'll be something we'll be super curious about to get into the analytics Awesome.

Okay, Christina, thank you so much for sharing. That's an awesome, inspiring story. Thank you.