Managing Project Metrics | Katie Robinson, Principal, Marketing & Communications Officer, LS3P

Hello, welcome in. So as you trickle in and get settled, I've got an opening question for you in the chat. That I'm about to paste. And the question is, what's the best thing you've learned how to cook during the pandemic? And I'll chime in with my answer.

Ooh, dal.

Dal's the best. We cook a lot of that too. Cauliflower crust pizza, we did that last week. Nice. Dolma, Bruce, I'd love to learn how to do that. Looks like we've got some smoking, smoking of things at home. Some pad thai, love it. Masala chai, delicious. Good stuff. Air fryers. Yeah, I've been seeing a lot of those going around. More tofu. So great.

Cool.

Good.

So we've got folks still coming in, but I'm going to go ahead and get us started. Please feel free to keep that going in the chat. Hello and welcome to this deep dive on managing project metrics featuring, featuring Katie Robinson of LS3P. I'm Christopher Parsons, founder and CEO of Knowledge Architecture. So today's structure and logistics— for the first time this year, we are having a presentation on a deep dive. And so Katie will start with a 25 to 30 minute presentation about managing data metrics at LS3P. And then we'll have dedicated Q&A time at the end. And so for those of you that have used the Zoom webinar format before, or for those that haven't, there's a Q&A panel down at the bottom. Please feel free to add questions as we go, and then when we come back to Q&A at the end, we'll just kind of proceed through the questions in order of the number of votes they've gotten. Um, the deep dive portion, which we're doing now, of today's program is being recorded and will be available after this event. For those of you who signed up for the 60-minute hangout afterwards, where we're going to go into small breakout groups, we'll be leaving this webinar and then heading over to a separate Zoom meeting so we can use that breakout feature. If you didn't sign up for that hangout but you'd like to join us, there's plenty of room. Don't worry, you're all invited. I'll just share a link at the end of the webinar that you can use to join us. Sarah Davies and Catherine O'Leary from our team are going to be taking notes during this webinar in a Google Doc. We call this collaborative notes. Please feel free to jump in and take notes alongside them. So you can capture quotes or links or resources or anything interesting you think that happens and put it all in one place, and then we'll share that out at the end. So here's a, here's a link to that doc. Oh, thanks Sarah, you shared it too. Now you have twice as many chances to click on that link. So that's it for logistics. Let's get into our conversation. So as I'd like to start by introducing Katie Robinson. Um, so as LS3P Principal Marketing Communications Officer, Katie Robinson plays a vital role in the firm's practice and business processes, working closely with the executive committee to ensure that the firm's goals are fully supported. From a practice standpoint, Katie manages the firm's internal and external communications, including all media platforms and digital presence. And as part of the firm's business team, Katie leads LS3P's knowledge management efforts. Um, I— hi Katie, thanks for joining us. Um, I've gotten the pleasure to know you over the last several years, and I've always found you to be thoughtful in all things AEC marketing, business, knowledge management. And when you first told me about that data manager story, I knew I had to share it with the Kay Connect community. So I'm happy to welcome you to Kay Connect, and the stage is all yours.

Thank you so much, Chris. I'm so happy to be here. I just love this group. So like Chris said, we're going to start out with a presentation. So let me quickly share my screen with y'all. Okay, so I'm going to start by telling you a little bit more about LS3P. We're an architecture, interiors, and planning firm, and we were founded in 1963. And we are really, really guided by our vision, mission, and values. And I know a lot of companies and a lot of firms say that, but we truly, truly, truly mean it. And as we dive into this presentation about our Data Manager program, there's a few keywords from our vision, mission, and values that I really want you guys to keep in the forefront of your mind. They are expertise, collaboration, empowerment, and proven results. So our commitment to these, these words that are in our vision and mission and value are embodied by all of our team members. We have 339 team members. Located in 8 offices throughout the Southeast. Just to give you a little bit more background on the firm that's relevant to the data collection efforts, we have roughly about 750 active projects, and we do utilize Deltek Vision. So that just gives you a little bit of background for our firm. I'm often asked, you know, you're in marketing and communications, why are you talking about data collection and data management? Because it's not exactly creative in nature, right? It's kind of a little bit more mundane. But I'm a firm believer that in combining data and analytics with marketing and communications, you can create incredibly impactful marketing pieces that help to build brand trust. In our industry today, it takes more than just a pretty portfolio. Clients want facts and figures, and in order to give this to them, you first need to gather project data. So I'm going to kind of go into the idea behind our program. I joined the firm in 2004, and at the time, all I knew about Vision is that I used it to do my timesheets. My world was proposals and interviews and general marketing type activities. So when we were going to do proposals at the time, we did what most firms do. If we had to answer any sort of question that pertained to project data, we first looked at past proposals. We looked at tons of Excel spreadsheets that were saved in a variety of network drives. We had these great three-ring binders printed full of project data that we could flip through and try to come up with some sort of answer to the questions that we were getting. Or my favorite thing that we did is we went to our project managers and we asked them a question about a project they worked on 10 years ago, because that's like really fun to do for our project managers, right? They love digging deep and trying to remember things that happened 10 years ago. Needless to say, when we asked these questions, we were met with a lot of frustrations from our project managers, even the most patient ones, because not only are they stopping working on active projects to answer these questions, 9 times out of 10, they've probably answered these questions 2, 3, 10 times before in the past for the same type of submittals. In addition to proposals, things like simple surveys were taking us tons and tons of time to submit for because we didn't have this data at our fingertips. So we knew that something really needed to be done. So we started thinking about different ways that we could go about, you know, collecting this data and storing this data. And we were thinking, if we only had someplace to put it, if we only had a database, Well, naturally we did. We had Deltek Vision. We just weren't utilizing it to its full capabilities. But most importantly, we also asked ourselves, what if you get someone to manage this data? So of course, the majority of the people at the firm looked like the individual on the screen. You know, nobody really wanted to step up and take on this enormous obstacle that we've been facing for so long. But as the old saying goes, don't complain, about something unless you're willing to be part of the solution. So I, along with some individuals from our business team, started to outline a solution using Deltek Vision. Looking back, this was really when we took our first steps towards knowledge management at LS3P. So the first step was determine the goals. How are we going to use the data collected? What types of reports would we like to run? And our mindset is, if we were not going to report on it or use it, we're not going to ask people to collect it. So then we really evaluated Deltek Vision field by field, literally guys, field by field. We opened every single one of them. If that field did not support our end goals, We eliminated it. So we took it out of Vision. We either moved it to another tab and hid it from everybody, or we removed it completely from the database. So once Vision was organized, we were ready to begin populating it. So we turned to the most natural people in the firm to do this, our project managers, because we figured they had nothing else to do. So seriously, we tapped the, some of the most busiest people in the firm to also be in charge of managing project data. We did provide an overview of the data we were hoping to capture. We ran reports, and then we sat back, and we waited with anticipation to see what will happen and what type of progress we were— we made. And we did make progress. We made very, very slow progress for years. Using this method. So some data began to trickle in. However, proposals and industry surveys were becoming a lot more complex in the type of questions that they were asking about projects and project data. So we knew we needed a better system to, to collecting this information and getting it into the database so we could report out on it. So in 2014, we had an idea to engage another group in the firm. We wanted to engage our emerging professionals to help the project managers capture the project data. But first, we had a little bit more work to do. We had to change the mindset. No one group in the firm owned Vision. A lot of times when I talk about this program with people, I hear, "Well, accounting owns the database," or, you know, "The tech team owns the database." really tweaking that mindset that the database is no one person's responsibility or one group. Accounting has responsibilities, as does marketing, HR, technology, and the practice team. So we changed that mindset in the firm. And then all team members were given access to the project records in Vision. So the Project Info Center. With some map restrictions there, but primarily they had the majority of information. And then we used field-based security to determine who could edit project records, because we really wanted to make sure that we maintained data integrity. And while everybody could see all the projects, not everybody could change the data on the project records. We also set up a unique project number, and by unique, it— really wasn't that unique. It's LS3P Vision, so we didn't really go out on a limb to name it. But the reason that we did this is because we wanted individuals who were helping to gather this data be able to charge their time to a particular project number. This helped for a few things. We could track the success of our program and how much time was being spent on it. But we also needed buy-in from everybody to make this successful. So if we went out and said, okay guys, we're going to put together this new program, we're going to capture all this project data, and guess what, we're going to charge it to the project numbers, project managers would not have been on board with this program, because then they're like, really, one more thing that's being charged to my project? I don't think so. So we really needed to take the approach that this was going to be an overhead expense, but we also wanted to make sure that people's utilization rates weren't impacted. By doing this. We needed to be able to measure how much time was being spent on this process. So now we kind of had everything in place, and we had to see if it was going to work. So we pulled a report of closed projects for the past 5 years and asked our associates to try to gather as much data as possible that was missing for those projects in a 6-month timeframe. So we had about 55 people ga— gathering data on roughly about 2,000 projects. And we really only focused on dates, like design and construction completion, cost, both estimated and actual, um, size, project type, like sector or construction type. So we didn't focus on all of the fields we wanted to collect information on. We kind of took a little chunk out of it and just focused on those. And we ran reports at the beginning, so we knew where we started from, and throughout the process. Well, about 4 months into this effort, we had a specific question come up in one of our marketing proposals. And the question was asking about our, our history in the past 5 years with a particular type of project for our K-12 sector. So at the start, At the start of this process, if we looked back at the report, we were able to say that we had 6 projects that would be similar to the size and scope of the one that we were pursuing. Not great experience, not horrible. At least we did have a couple. So we decided, even though the 6-month process wasn't completely done yet, we ran another report just to see what are we looking at, at now. So this was April. So 4 months since the start of the process, we went from 6 to 30, which was a significant difference when we were submitting for this proposal. At the conclusion of the 6 months, the associates were able to gather about 80% of the missing data from closed projects. So it was much more successful than we thought it was going to be, and we were really ready to formally launch our data manager program. So the first thing to note is it's a role, it is not a position. So we did not hire more overhead individuals to do this. It is simply a role, just like any project team role. So you can have a principal in charge, a project manager, project architect, and so on and so on. And then you have a data manager. So every active project was assigned a data manager. We added a field, an employee lookup field, in Vision on the project record so we could really assign somebody. And the project managers were responsible for doing this. Their data managers are typically emerging professional level. Sometimes they are associates or senior associates. On smaller projects, the project manager themselves may serve as the data manager as well. If the data manager rolls off the project, you know, some emerging professionals may go from project to project once their role is done. Then the project manager will simply replace them with somebody active on the project team. Our amazing technology team also took those individuals that were listed in the fields and were able to automate an email group based off of those, and those were really key because that was our source of communication at the time for training. So we have all the individuals entered for active projects. We pulled them together for some virtual training. And we went field by field and let them know what we were trying to accomplish with this program, what we were looking for, where to find it, and, um, and what we were planning on doing with the data. So we also created our intranet that has step by step, step again, every single field that we collect, why we collect it, where you can find the data, and how it correlates to Deltek Vision. And then we held constant check-ins, automated reporting, especially throughout the first year to make sure that the data managers were keeping up with their projects, didn't have any questions, didn't need some assistance. And we, we made some progress. At the end of the first year, we were able to say that we gathered about 20% of project data. Now, this does not sound like much, but in years past, if we gathered 10%, we were doing really, really well.

As well.

So we all but doubled what we had done in the past. Um, we spent roughly about 212 hours. Now you're going to notice there's an asterisk next to that. So like many firms, you can create a project number. Making sure people use that project number is a whole nother story. So my guess is you could probably inflate this by between 50 and 100 hours, um, based on what, really was put into this effort. But we did, we did make progress, which was, which was our goal. In 2019, we started seeing a significant improvement, getting to where we had 90% of project data at year end on our project. Again, you'll notice the number of hours, not a significant investment to achieve this 90% mark because it became part of just the everyday practice of the data managers. Reporting started moving to more of a monthly, fewer training sessions were needed because individuals were getting used to doing the data entry. So, the program began to run, run a lot smoother. In 2017, we also, in addition to gathering project data on kind of the surface level, the easy-to-get statistics like construction cost and things like that, we did targeted approaches as well. So we took a look at things that happen throughout the year with us. For example, AA 2030, professional surveys that we submitted for, professional liability insurance renewals. And year-end information that we were trying to collect. And we set quarterly targets to collect data above and beyond what we are already asking the data managers to do with quarterly effort. We provided training for all of these. So again, communicating why we're doing this, why is it, why is it important, how are we going to report on it. It was just training was Very, very, very essential to all of this. These quarterly efforts were incredibly successful of us being able to really dig deep into breaking out project components, understanding more about cost per square foot, being able to report out. They really were tremendous, and we were on track. Every quarter we were doing these. Then 2020 happened. So this year we haven't done as many quarterly efforts as we had hoped. We got through the first quarter and then COVID happened. So we held off on a lot of the quarterly efforts while our team members went to work from home and had that adjustment. And now we're kind of in a hybrid with some individuals at home, some are in the office, and some kind of bounce back and forth. So we really did not want to press during this time. We wanted to make sure that our team members had that time to adjust. So what we did is we didn't hold off on our knowledge management efforts by any means. During this time, a lot of our professional— administrative professionals did not have enough work to sustain 40 hours a week. So We brought them into knowledge management. We dusted off some of the old Excel spreadsheets that we had. We even found books of data from years and years and years ago. And we adjusted the admin security role to give them access to edit the project records. And then we trained them on how to enter the data into Vision. This was so amazing. Because in a 3-month timeframe, they were able to enter data into over 5,500 fields, closed projects. So 5,500 fields they found and entered it into past projects. More importantly, we were able to keep an amazing group of professionals working full-time and engaged. With our gradual return to office, we've actually kept them engaged in entering data from past closed projects. We've found more spreadsheets, because there's always spreadsheets floating around. We found more spreadsheets, more old file folders going around, and they've jumped in and have continued becoming a tremendous addition to our knowledge management efforts. And our knowledge management team. So I've talked a lot about the program, and normally when you get to this point, people are like, okay, yeah, that's great. What's this going to cost us, you know, for the firm? What's the bottom line? I'm not going to give you an exact figure. A lot of these benefits, they can't be measured in dollar signs. What I can tell you is we have seen about a 20% reduction in the time it takes us to produce proposals. We have about an 80% reduction in the time it is taking us to assemble the professional liability insurance renewal application, which used to be a tremendous process for some members of our business team. And we have a 90% reduction in the time that it takes us to complete industry surveys. So the product— if I just look at the proposal of reduction, that's just for the marketing team members. So that does not even touch on the project manager's time that they're saving where they don't have to answer the same questions over and over and over for marketing to be able to produce their submittals. So that time saved is also a tremendous, tremendous advantage.

For our firm.

We also see a lot of increase in data integration from Vision into other applications, such as intranets, planning software, to websites, digital asset management applications. All of these benefit from an increase in project data in Vision. One example, we do utilize OpenAsset for our image collections. So we're able to pull linked fields from Deltek Vision into OpenAsset where we can have greater project filtering. So we can filter by location, by type, by size, even by cost because the data is in Vision. We also saw a tremendous increase in our internal reporting capabilities and taking data from Vision using a data visualization application to, um, to manipulate this data and, and to come up with some really tremendous reports that we use a lot for strategic planning on an internal level. But the most impactful outcome that we have seen is the individuals who are serving as data managers becoming nexsperts, or that next generation of experts. These emerging professionals are being exposed exposed to things that the older generations weren't at their time in their career. They are looking at owner agreements, they're looking at consultant contracts, they're understanding cost per square feet, they're understanding different fee structures by sectors, by project type. So they really have an opportunity not only to gather the data but to really consume it. That outcome, in my opinion, is beyond any financial measure that you can put. So this is all really, really great. And you're kind of wondering, where do I start? And that's where we were many years ago. We had these grand ideas, but you're like, oh my gosh, there's no way. This is just too much. So the first thing that is really important for firms to consider is find your person. So find your data geek, whatever you want to call them, just find someone who can really dive deep and get passionate about this project. This does not have to be a new hire. It does not have to be a full-time position. So I kind of assumed this role, and at first it took about 25% of, of my time. Not a tremendous investment for the outcome that we've had. So it is completely manageable to do in correlation with another position. And another true key is you have to get support from the entire leadership team, all the way to especially the project managers. It will not be successful if your PMs are not on board with it. So if you need something to convince them, Start small. Do something like we did with the past 5 years of experience. Find something that is kind of a splinter in your side, a question that y'all always have to answer over and over and over, and tackle that first. Let's say it is you just need to know when your projects were completed. Simple task, but a lot of times firms don't have that information on file. There's several reasons why that's really important to have in an easy-to-grab database. You know, one thing could be just to track the time where you could potentially have lawsuits from the time a project ends, you know, to the time that that expires based on the state that you're in. But also to do those reports like how many projects have you done in the past 5 years, 10 years, 20 years. So start small. Take that field, 3 fields, 5 fields, go back 1 year and see how much data you can gather and track your time. And then show the difference using before and after reports. This is one way where you can start, uh, convincing folks of what benefits could come from having the data at your fingertips. So once you get that approval, still keep it small. Don't jump into the entire program. Do another test, maybe go back 2 years, maybe go back 5 years so you can ease people into it. Make it part of your culture and make it part of onboarding. Every single person who joins LS3P, whether they are on the practice side or business side, they are onboarded in data management and the data manager management program. So everybody gets as some sort of exposure regardless of the level you come into the firm. You get exposed to what we're trying to accomplish with this program. And then communicate a lot, and when you think you've communicated too much, communicate more. So constant reporting to leadership on how the program is doing, constant reporting among the project managers and the data managers, checking in with folks making sure people know why we're collecting information and why and how we're using it. All of these things go to just help reinforce the fact that this is part of your culture, you are passionate about it, and it is something that the firm stands behind. And then celebrate. And this is the one thing that I think people overlook the most. You know, when you have, if you do a quarterly effort and you get the results, share them, share why it's so great that we have this new information. You know, celebrate with your, your data managers, your project managers, make sure leadership knows what they're doing, and thank them for the amazing job. You can also find ways to incentivize the program. That's what really perks up people's ears a lot. And this is, of course, up to individual firms to see what works best for you. Maybe you can work it into your model for bonuses. Or your merit increases. You can take a look at the amount of time that's being spent on that unique project number throughout the year, and if it looks like that time is going to impact people's utilization, maybe the firm decides to roll that back into utilization so it's not a negative towards, towards the team member. Find what's comfortable for your firm, but make sure there is some some sort of incentive tied to this. And then keep people engaged, keep them excited, and celebrate their hard work. That is the just key and essential thing to making this program so successful, is to make sure that people are truly appreciated for all of the great things that they're doing to, to bring this data in to help to help increase your design portfolio, to help increase the metrics you share with your clients, to help increase your marketing materials, to go from a pretty portfolio to producing materials that are trusted and really add value to your clients. All of that is a result from having the, the background of having the strong project data in your database where it's easy to pull. And our data manager program has been what's made that so successful for us. So at this point, I'm going to pause and see if there's any questions.

All right, Katie. Thank you. Actually, if you don't mind stopping sharing your screen, then we'll do that way. Cool. So I'm going to start by taking questions from the community, and I'll just fold in some of my own questions in the middle of that. So for folks that have questions, or if you want to kind of see what other people are asking, go ahead and click on the Q&A tab. You can upvote by clicking on the thumbs up next to any of the questions you really want to hear answered. And I'm just going to go top to bottom in terms of answering questions. And so our first question is from James Farakhor. James, I'm going to go ahead and unmute you so that you can ask your question. It's the one about data managers and who are good candidates. I see you have multiple. Can you hear me?

Yep.

Yeah, so I was just kind of wondering if you had any, I guess, recommendations about what you did with data prior to having Deltek, because one of the problems is that there's information that we like to leverage from older data, but it's not in Deltek. So I was just kind of wondering what your plan was for managing past data or archive data.

That's a great question. So we have, like I mentioned before, we have data everywhere still, even though we've had the Data Manager program in place for several years now. There were a lot of years that spanned before that in our firm. So if you're like us, we had countless Excel spreadsheets, we had literally binders, guys, full of data that would have been printed off from various other— we'll call them databases in the past. My guess is they're probably Excel spreadsheets in the past. So that's why we've utilized some of our admin team to help put in that, that information. Even if they— we didn't have project numbers before, we can create project numbers and put them in Vision for closed projects and then just automatically make them closed and enter the data there as well. We also— we merged several times throughout our firm's history, and one of the firms in particular who we merged with went back from 1922. So there was a lot of data there. Now, I'm not going to say that we went through and put in data from 1922. We did not do that. So really right now what we have focused on is 20 years, knowing that when we do a report, if somebody's asking for our complete history, We'll run the report and then say, this goes back 20 years. You know, the firm has been in existence for 57. So we kind of do that disclaimer as well. But we have tried to collect as much past data as possible throughout this program.

Great. Thank you, Katie. Actually, James had a second question that I'll just ask on his behalf. You mentioned the data managers. Met multiple times. He was curious about who good candidates are to be data managers for project teams. Are there any kind of recommendations for skills and qualifications that would make someone a good data manager?

There's no true skill set needed. I hate to say that because it is such a great thing. But essentially, we really look at the individuals who are just starting their career, who are probably tech-savvy. Tech savvy, because you do have to have a little bit of that element to you, and just who are curious and want to learn. We really try to get every single emerging professional in our firm to serve as a data manager so they can build this experience and knowledge, whether they choose to go the project architect route or the project management route or another route, you know, besides those two. We really try to give everybody that experience in it. And like I said, it really is— we spell out the entire program. So it's very simple for anybody to jump in and be a data manager. Awesome.

Thank you. Our next question comes from Leanne Becker. So Leanne, I'm going to go ahead and make you so you can talk.

Okay.

Can you hear me?

Yep.

Okay. I'm just wondering, looking at the voting, it looks like I'm not the only person that has a system where we have multiple project numbers potentially for the same project. Some projects just have one and some— I think our record is one project has over 300 different Vision project numbers that relate to the same overall project. And I mean, that's obviously an extreme example. But there, it's not uncommon at all to have more than one commission number, and our numbering system is based off of the primary billing client, and so if you've got a situation where you may bill more than one entity for the same project, you know, how do you record that data? And we did come up with somewhat of a solution. We just— it hasn't— It hasn't really 100% taken hold yet, but it's, it's definitely gaining some traction. And we, what we've done is we've identified a parent. Every commission number will be either a parent or a child. And so child records roll up to the parent and regardless, and you know, it doesn't affect any of the billing or anything like that. It's really just for data. And then in that parent, that's where we record the, you know, the overall metrics rather than, you know, just each individual project number. And I'm just wondering if anyone else has any other ideas. That's the best we could come up with, and it probably— it took years even to come with that after, you know, surveying people across the entire firm about how to manage this, because it was keeping us from recording anything.

So if it's other people than Katie, you can put your answers in the chat. And then Katie, I'm interested to see how LS3P is tackling that.

Right, right.

So we, we don't have 300 that are the same. That's insane. But I will say this is where the mindset that vision is owned by multiple groups really comes in handy, because one of the things that we did, we needed to make sure is that our accounting team are the ones that enter the project numbers or create the project numbers. I think a lot of firms have it set up that way because then you can tie everything directly to the billings. So, you know, we had to make sure that when they were entering it, they entered one project, and then we use tasks if it kind of branches out and maybe there's some added value services or something like that that you put on there. We'll use various tasks within it, which sounds like it's kind of that parent-child thing that you have going on. So, you know, that's one thing. We also ran into where a project would start as a study, but then transition to a project, and sometimes we were taking out a different project number, and sometimes we weren't. So our data managers know that if it's a study and you get to a certain field, you don't have to— there's nothing else to collect because it's a study, so all the other fields you can ignore. So we were running into an issue where a lot of the projects started as study, so they were marked as study, but then transitioned to full projects, and we weren't collecting all the project data because we were like, "Oh, it's a study, I don't have to do anymore." So not exactly the same thing as you're talking about, but it was kind of duplicating the reports for us when they were two different numbers. Because we're reporting either more experience or we're reporting the same project on professional liability renewals and things like that. So working with your accounting team, it sounds like you kind of have something that you're going to, to try out. Just when you run your reports and everything, make sure that everybody understands the data goes into the parent record, not the child records, and then you can report maybe there's some sort of field that specifies it's a parent record, and then you can make sure that you report only on those parents. So you're not getting those duplicates. Because my guess is, even if you tell people, "Don't put data in any of the child records, only the, the parent one," people are going to still put it in the child one. So just make sure that your reporting matches up what you decide to do with those potential duplicates.

Great. Thanks, Katie. Next question comes from Camille. Camille, I'm going to go ahead and unmute you.

So, yeah, just looking at the project closeout process, you talked about, you know, final billing, all of that. What does that look like at your firm? So I can tell you what it looks like from the data perspective. So, for example, we're in that process at the end, or the last quarter of every year, we do our year-end data collection efforts. So we take a look at all projects that were closed out by accounting throughout the year, and we make sure that we have all the data needed for, for that project. So we send it out to our data managers. Luckily, it's a lot easier now. When we first started this, they were like, oh my gosh, this is crazy that you're asking us for like hundreds and hundreds of fields of information, but now that they've been entering the information throughout the life of the project, the year-end collection is really a review, and maybe there's 2 or 3 things missing. What we did add to it is we added a field in Vision, and y'all are going to laugh at this, but it says, you know, is this project, is the data in this project record as complete as possible? And it's a simple yes/no. Because sometimes, y'all, there's just things you cannot find. You just will not get that data. And we didn't want to keep hitting the data managers for this information and asking them for it if we knew it couldn't be found. So once a project is closed out and they've exhausted all resources, they've put all the data they need, they can select that as a yes, and it removes that project from any reports going out to request project information. So that was the big thing that we learned, because at first we did not have that field And people were like, guys, I've already told you 2 years ago, I don't have this information. And it was very frustrating for them. So now the caveat is if the project status is still active and they try to mark that field a yes, it will not let you. So we did do a lot of field automation where you couldn't accidentally click that or get frustrated and just want to click it. So there, there are a couple of caveats, behind it.

That's great.

I—

we got a question. I'm going to try and mix between some write-in questions that we got too with registration, and one that I saw a lot of different flavors of is, you know, you had a slide, the Power BI slide, where you showed your different market sectors. And so how do you think about metrics that are relevant for all project types and then some that are market sector specific, and how does that play into the Data Manager Program?

That's a great question. So that's what we're kind of tackling now. So the data that we have asked for so far could apply to any of our sectors. We have about 12 different market sectors that we serve. So we're talking about things like breaking down project components, using different SF 330 codes to break down fee, construction information, square footage, things like that for project types. We have had some requests from like our healthcare team and our K-12 team that say, oh, it would be great too, because people are getting excited about, you know, all this data, if we could collect XYZ and all that. So what we want to do is we want to be very hesitant where we don't overpopulate Vision where it becomes confusing. So typically we go back to those teams and and say, what are you trying to accomplish? Tell us what you're trying to accomplish with this data, and we will see if we can make the existing fields work for you. 9 times out of 10, we can. If it's something ultra-specific, we tend— for example, a lot of people want to break down the various components by square footage in their project. Well, in Deltek Vision, there's a Ratios of Interest grid, and it does just that. You can put however many breakdowns you want in there. If you're doing a hospital, you can do it down to the types of, you know, disciplines that are in that hospital, surgery rooms, laboratories, anything that you want, and note the square footage of it. So I would say first take a look at what's in your database, whether you use Vision or not, see if it'll answer the questions and the end results, and hopefully It can. If not, one thing that we have considered doing is creating individual tabs for our primary sectors, not all of them, maybe like the top 4, and say, in addition to all of these, for K-12, we would love to get this. For healthcare, we would love to see this. We haven't made that step yet because we found that the reports that we're already generating and the information is serving our purposes right now. Not to say we won't in the future, though.

Got it. Yeah, that's great. And kind of following on that question is, beyond the metrics— this comes from Nicole at TCOM— how do you capture project stories, as well as kind of more of the harder data points?

Well, that's a hard thing, isn't it, guys? That's when marketing and data don't collide sometimes.

Right.

But to be honest, we do have a marketing tab in the Project Input Center. So every project record utilizes the marketing tab. In an ideal world, everybody puts the big idea of the project, kind of the what, what we hope to, to do for the client. That's where they would save the description, the marketing description for the project, and maybe awards copy and things like that. I will say it's hit or miss. We do not make this a data manager responsibility because, quite frankly, the big idea of the story should fall on the project manager's shoulders, not the emerging professional. Marketing enters it when we know it and we can. Now that we do the integration with OpenAssets, people are seeing a lot more value to putting this information in because it does pull the marketing description from Vision into OpenAsset, right? So, you know, proving that having this information in there helps other applications is letting us gain ground on it. But yeah, trying to capture that big story, you know, if anybody has the magic bean that does that for us, please share it with me.

Yes, put all magic beans in the chat that you have. Awesome. Eric has a question about AIA 2030. Hello. Hey Eric. Hi, how you doing? Yeah, I think you mentioned AIA 2030. I was just wondering, as part of that, do you currently track like project sustainability metrics or modeling software, any energy things like that in the Deltek platform? Project Hub, or do you do that elsewhere? We do.

And actually, we are just this week in the process of cleaning that up. We have a fantastic sustainability leader in our firm who really took a deep dive into the fields that we were asking for and made some suggestions to us of how we could tweak them and make them a little bit more consistent with what the— not only the AIA 2030 is collecting, but also where we want to take our high-performing building metrics. So we're currently kind of rearranging those tabs to make them suit our needs. We do have an in-house knowledge team that focuses just on sustainability, so different folks in our firm will roll on and off of this knowledge team throughout the— like every other year. And we can send them reports from these fields and make sure that all projects are being captured. And then if we do need to engage the project— the data managers in collecting more, we can. I will say, whenever we decide to change fields in Vision, it's a big decision because our folks are used to seeing one thing. We don't want to rock the boat, so we make sure that we pull reports on all the data that's currently in the fields. Now if we pull it and we see that only 2 projects have things in there, it's pretty safe to assume nobody uses those fields anyway. But if, if there's a lot of data in it, we make sure that we preserve that data. Maybe it's just we want to tweak the field name because something changes and it's not as intuitive as it, it used to be. We make sure that we create a new field so the I don't know if y'all are like us, we had, um, the label in Vision was changed quite a lot, but the field name wasn't. So it looked great in the database until you went to run a report and then you're like, oh my gosh, none of the fields are here because they're named something different on the back end of Vision. So we made sure we created fresh fields. We did a find and replace to merge any data into that fresh field. And then we built from there. So I would say, you know, any data, again, whether you're reporting it to the 20— AI 2030 or reporting it for other things, make sure that you are going to use it for multiple purposes. And that way you can get the biggest bang out of your buck. But don't just willy-nilly go and change fields all the time, because then people will lose faith in your database really quickly.

Katie, I gave a presentation with one of our clients called— and we called it return on information, which is what I think you're getting at, right? So it's a different ROI, but like, if you can raise the value of that single data point across 4 or 5 different platforms or different report, you know, every time it goes up, you're more likely to get it captured, right? Um, Christopher, uh, Tyndall has a question about automatic data capture. Christopher, you might have to unmute yourself. Maybe we lost—

I found it, I found it.

Ah, you found it, great.

Sorry guys. The question is, uh, how much of this data is captured automatically, uh, as opposed to stuff that has to be manually input by your data manager or the PM or the admins or your marketing team? And then also a kind of a follow-up question on BI and if you're starting to use business intelligence to get better insights from this data.

Christopher, can you say a little bit about what you mean about captured automatically?

A lot of the data could be captured automatically as, you know, say you're going to set up a project and your initial project setup number request could require a square footage or a project type, which then is kind of a manual input, but it's kind of captured automatically. It doesn't have to be captured after the fact. Although maybe the project square footage changed, so then that then needs to be updated after the fact. I mean, there's processes that can be done automatically.

So we do have some that's automatically captured. We also utilize the Opportunity Info Center in Vision for any pursuits that we're chasing. Part of that is listing the basic elements of the project as we know them at that time. Like you just mentioned, they often change, you know, after the project is awarded, sometimes 5, 6 months, a year after the project is awarded. So what we do is we put as much possible in there. We also changed the procedure to requesting a project number, because just as much as we want to report on our projects, we want to be able to show statistics and analyze our opportunities and our pursuits. So we wanted to drive people to use Opportunity Info Center more. Before, our project number request form was, I think, a Word document or, some sort of automated thing, but it was out of Vision. You did not do it through Vision, and then it went to our project accountants, and they set up a project number for you. In our Opportunity Info Center, every opportunity record now has a Project Start tab. So you click on that, you complete that field, and then when you hit— it says something like, are you ready to send this request to the project accountants? You say yes, then they get the automated email saying, "So-and-so has requested a project number from this opportunity." So then they take the opportunity record, and they map it to a new project in Vision. So all of the data that we have deemed map-worthy will come over and map. When an announcement goes out automatically to the project manager— and we also ask for a data manager at that time as well. So the project manager and data manager will get an automated email saying, hey, congratulations, your project number has been activated. Please take a moment to open it and verify the information that came over from the opportunity, just to have another set of eyes, because we do put estimated cost and things like that in there. We just want to make sure that the data being pulled over is as accurate of a start as possible. So we do do a lot of that kind of automation in terms of the basic information on the project. Your second question was about data visualization, I think, and Power BI. Was that correct?

Yeah, that's what I've got in my notes.

Cool. So I am not doing commercial for Microsoft here, but y'all, Power BI is amazing. I mean, it is just changed our world in how— I mean, we're a visual industry, so if you've ever tried to do a graph out of Vision or just download a report, they are not visual. So it was kind of a painful process for us getting people to be really excited about the information when it looked kind of dull coming out of it. So now we can run reports out of Vision and without manipulating them, save them to Excel and then just refresh a Power BI, and it takes a matter of minutes. They're filterable. The, the data is amazing. It's so helpful. Like I said, we use it in our strategic planning. Often we use it on a weekly basis for various groups in the firm. So Power BI and its integration with, with Vision is next level for us. And that's what we really dove into deep this year.

Awesome. I think I'm going to read a couple ones just to— we're getting close to the end. I think we can do one or two more. Elise wants to know about VantagePoint for LS3P. And so is that happening in the next year or two? And kind of maybe a follow-up for me is how does that or doesn't that change your thoughts about data management going forward?

So we have the sandbox of VantagePoint right now. So any of you who have played with it, you know what the sandbox is. They essentially take your existing database and make a copy of it into VantagePoint, and it shows how things come over and all that loveliness. We did a lot of customization to Vision, a lot of customized fields, so it was really interesting to see how those pulled over, and they did. They pulled over very successfully. They're just a little bit all over the place right now. So I will say we do have plans to move to VantagePoint. We are not going to do that. Probably— we'll probably start looking at it and do another sandbox in 2022. But I've participated in a lot of conversations and webinars and the like with Deltek and VantagePoint, and it's not going to impact this program. So we will continue to do the Data Manager Program. In fact, we'll be able to do kind of custom dashboards within VantagePoint for the data manager. So when they open VantagePoint, they see their projects, they see charts and graphs made from their projects. So I think it's going to be a really powerful tool for us. We're just not going to— we're not going to jump into it right yet.

Yeah. We squeeze in one more? I think we can try and do one. James Van De Zand, you get the last question today.

Oh, how exciting.

Okay.

Yeah.

Hey, Chris. Hey, Katie. Great presentation. Thank you for sharing all that. The one thing I— at the risk of diving way down in the weeds at the very end of this, I was wondering if you had any— could share any notion of like what are some of the most valuable data points or metrics that you're collecting that you can see, you know, collecting and reusing Aside from project number, project name, what kind of project it is, like what are some like, you know, like, oh, these like 10, 5 or 10 things are really, you know, have changed how you're using the data and the metrics in proposals or even just kind of just interdepartment, you know, design research, let's call it. Does that make sense?

Absolutely. Absolutely. That's not digging too deep. I think it's good. So, from a marketing perspective, I will say being able to have estimated versus actual, whether it's cost, fee, square footage, change orders, things like that, all of that, having that at our fingertip has greatly, greatly impacted marketing and how we're able to respond to efforts. So, from a marketing standpoint, Having those comparatives of, you know, budgeted or before and actuals is tremendous. From a practice standpoint, if you think about having— let's say you have all of the consultant costs in your database, and you're able to run reports on the percentage of your fee that went to MEP, civil, structural, to understand fee percentage for your future projects. If y'all are like us, if we took a look at our fees for— from, you know, different consultants and everything, they could be all over the board. You know, they could be charging you 10% on one project and then 20% on the next project. So without data to support it easily at your fingertips, you have no way of helping those negotiations, nor do you really have a way of helping to train the emerging professionals of how to look at a project from developing the fee for a fee proposal and things like that. So we're able to use data for the business part of our firm, as well as the practice part of our firm, as well of really truly understanding the metrics and how they could advance our profession and allow us to be better stewards of knowledge for our clients. So that was key for us.

Katie, thank you so much for making time today and sharing your journey and being so open and transparent with us. I always love— this conversation, you can tell by the chat and the level of questions we didn't even get to, we could keep going for a while. So thank you. Everyone else, thank you for coming. We're going to send the recording, the slides, the chat log, and a link to the collaborative notes out tomorrow. For those of you that are sticking around, we're now going to switch into the hangout with Katie. So we're going to be able to work through some more of those questions, but also give you a chance to connect in small breakout groups. So here's the link to the hangout space. I'm going to paste it in the chat. Before you click on it, you should know that once you click on that link, you'll see a prompt, and it's going to ask you if you want to leave this webinar and join the hangout video call, and you should say yes. You'll be there for about 60 to 90 seconds before the meeting starts. If you have any trouble at all, please email us at connect@knowledge-architecture.com. Okay, here's the link, and if you registered before, you should have the link in your email also, and we'll see you over in the, in the Hangout space. Thank you again, Katie. Alright.