Confluence S2E1 - Bringing AI to the Synthesis Intranet Platform

Welcome back to the Confluence podcast. We have Chris Parsons joining us again. Hopefully you've tuned in and heard the first episode that we did with Chris a few weeks ago. But if not, you might want to go back and listen to that. But happy to have you on again, Chris. You know, I know we're gonna, we're gonna dig more into the synthesis platform and, and really start to dig into, you know, not only what, what synthesis does, which I'll get you to kind of kick this off with, but ultimately like why, you know, how AI is affecting what you're doing and how you guys are thinking about that. So maybe you can just give us a good overview, introduce yourself and a good overview of what synthesis is.

Sure. So for folks who didn't hear the first episode, we started in 2009, we are 100% focused on AEC. Synthesis is an intranet platform for AEC firms and we work with about 130 firms, mostly us, although we have some international. And we are deeply integrated with a lot of AEC software that people care about. So Deltec, Uninet, Open Asset, New Forma, AEC 360 Zendesk and so on. So it's, it's Synthesis is I think, pretty well named product in that it like connects data from multiple places, giving you one kind of searchable source of, central source of truth. And I'm just going to do, I'm going to do like a high level seven screen kind of like look at synthesis. So people get like a, a kind of general idea of the kind of content that we have so that when we start talking about what we're doing with AI, it'll make a lot more sense. So I'm going to go ahead and share my web browser. So this is the homepage of Synthesis. This is a demo environment. So obviously you can brand it colors and fonts and logos and all the things to make it feel like home. But this will give you a general sense of what the product is. So most of our clients on the homepage will have an activity stream. So this will be posts and comments around different things happening in the company. So it's very much an internal communications platform. So whether it's a strategic plan update or a project tour or a new piece of software has been added or some pr, you know, somebody's retiring, you know, these are the kind of things that we typically will see on a synthesis intranet. And if I jump into one of those posts, you'll kind of see that it's, it's a very Rich multimedia. You can add images and videos and links and then we've got comments and likes and the kind of things you'd expect to see on a social intranet. If I go across the top, we've got our mega menu navigation and this is kind of just showing you the breadth and depth of content that typically will end up in a synthesis intranet. So kind of on home you'll usually find like whelping to the company and stuff like mission values, the leadership team, the different committees, different office stuff in an HR community, we call these communities across the top. So you'd find HR information, professional development, information, time and travel accounting, different office pages in projects, in practice, things like standards and codes, project management resources, different directories to projects in marketing, kind of brand templates, marketing resources, technology, the different platforms that people use and the technology policies and then learning, you know, different, you know, whether it's a company, university or different things like software training. So this is what we call a video library in synthesis. These have got different training courses depending on different platforms that the company's using. Video libraries have filters so you can add custom fields and kind of drill into those different videos and it's all searchable. So I'm going to skim across the top. So video is going to become important. We're going to come back to this later and we'll come back to this idea of a video library for training a little bit later as well. I want to talk about another key piece of content in synthesis which are document libraries. So if I go into our projects and practice community and go to our standards library, you'll just see that we've got some different standards here organized by CSI Division. And again you can add custom fields into synthesis. So it could be on CSI Division or standard type or conditions or whatever the different things you want to apply are. So that's videos and documents from a page perspective. I'm just going to go into technology and I'll go under our design technology section and go to Revit. And this is just a landing page in the company for all things Revit. So it's got links off to other different resources. We've got training videos on Revit in the upper right. I've got kind of our key points of contact. So I've got Brianne and Julie and then I can kind of reach out, I can see their teams status, et cetera. And then I can see any recent posts were written about Revit. So this is a pretty typical like. So if you imagine this kind of Layout. But for a lot of things within the company, you're trying to connect people to the right people and the right resources and helping them know how the company approaches, you know, whether it's a piece of technology, whether it's a policy, et cetera. If I go into projects in practice, a big kind of content type for us is what we call guides. So I'll go to the project management guide. And so this technology basically replaces like the hundred page PDFs that your firm might use for your employee handbook, or in this case your project management manual, or a CAD or BIM guide or a brand guide. There's all these different like kind of guides that we saw at our clients. And so we built a tool that handles that content really well. So you've got kind of the different people in project management. If I kind of go into contracts, you know, I can. You've seen we've broken what used to be a hundred page PDF is into a bunch of small chunks. And then these are all kind of like, you know, I've got a table of content. And I kind of jump through our different approach to contracts or billing rates or how we start projects or financial controls, like all the things, this is a nice place to put it within the Internet. So three more things I want to show you if I go to our project directory. So this is, I mentioned some integrations that we have. This is where those integrations with, you know, Deltec and UniNet and AC360 and open asset, Deforman, et cetera come in. It helps us pull information from those systems into one place. So I can do things like find all of our projects by type, you know, or by state. This is all searchable as well. Or I can put them on a map, right? And I can drill in and say like, well, what's the work we've done in California? And I can see we've got 11 projects in the Bay Area and then two of them are out in Marin, et cetera. So I've got a good way visually to kind of explore the work that the company has done. And if I click on one of these projects, I get a project profile. So this is data we're pulling in from all those systems. So I've got imagery from open asset, project type, cost and size, location descriptions, like a lot of rich data, as well as who worked on the project both inside and outside the company. If I click on one of these employees, I get to an employee profile. So this is our chief executive officer, N. Davison, and Again, I'm pulling information in from multiple systems. We've also got custom fields in synthesis you can use. And you can see things like professional bio and education and skills and her posts and what projects she worked on. And then if I go all the way back up to the employee directory, it's that same kind of idea with projects, right. I can filter employees on a variety of different fields and search for them that way. And the last piece I want to show you, because this is very pertinent to what we're doing today, is search. So I'm going to search on something like Revit. You can see that I've got some suggested search results, but if I hit return, I see everything. And so I see what we call best bets. So Rob Thompson's kind of our go to person for Revit. But if I keep scrolling down, I see best. We're searching across pages and posts and documents and projects and videos and all the stuff in synthesis. So that's kind of synthesis like at a high level, the kind of main moving parts that make up the intranet.

And Chris, the last time that you were here, you showed us this really great graphic that you've come up with, this periodic table where you kind of talk about the different. You can probably say it better than I can, but you have these categories with different topics within each category that are kind of knowledge centers within a firm.

And.

And that kind of shows that there's a lot going on. And you just showed, like, how all of that manifests in an intranet, but, like there's people responsible to get that content into the Internet. I always found that was one of the hardest things to do, was to get people to actually go and do those things on the Internet, to make it available for everybody else. Right. It is a step way above. And I know, Randall, you have a special place in your heart from a content management standpoint as well, of like, just making sure that the data is in there and that people can find it and it's accessible, but it still relies. You know, this is way better than files and folders on a. On a server somewhere. There's graphical aspect, there's layout, there's. You can do all kinds of things. You can link between pages much more easily than you can do between files on servers. You're still relying on people. And I think this is where we, we start to talk about the future of synthesis as well. Right. Which is the elephant in the room with AI. But up to this point, we've relied on people to do all of that work and you're showing this beautifully idealized version of an intranet. And then I think about the reality of someone's SharePoint intranet or whatever they've got going on, and half the data is on the right page and that person who did it no longer works at the company. And there's all kinds of other stuff. And I just want to talk about, I just want to throw all that out there because these are the realities of working in a firm where it's like, this is another job. Like, we need content managers now to make sure that this stuff stays up to date, that people are keeping an eye on the community, that it actually is grease between the gears of the firm and all these different departments and how they interconnect. And there's a lot going on there. But this is potentially where AI and things like that can actually help because I know you're going to talk about some more specific examples, but you can throw data at it and have it pull insights out of that in a much, I don't know, easier way. Let's just call it easier than it would be for someone else to comb through a document to split it up into all those different topical, you know, the table of contents that you showed in the standards area, there's video that, you know has been typically, if somebody posts an hour long video, no one wants to watch an hour long video. How can I find what I'm looking for in that video? There's a lot of challenges still in the content that even gets posted, let alone just the management of all that content.

I think you're right and I think we will dive into that. I think just at a high level. A couple ways I think about what you're saying are one, making the software easier to use has been a huge level easier to use. And then more beautiful the end product that you make with it. Right. That's been really important, especially we're 100% AEC and that matters to the designers and engineers in our community. I think the second piece is raising the ROI of having that knowledge captured in the platform in the first place. And so when you see what I'm about to show you with where we're going with search and AI, I think people start getting like, and this has happened already in our journey with synthesis, it's like, oh, I get it. So if we put our data in Deltec, that means I can get it out in synthesis in this beautiful way. Or the same thing with images from open assets. So by connecting systems, you raise the return on investment of that information. And so when you start seeing what we're going to do with AI, I think it's going to create kind of like a gravitational pull to get more content into the system because you'll realize, oh, if we have our best content, we're going to be able to do. And I know Randall and the folks at Avail are investing in this as well. You know, the better content that I get into my system, the further we can go as a company. And I think that's just going to be a step change from where we are today.

I'm going to make a comment, Kristina, and you know, I think you already made note that synthesis is a great name. You know, those names usually come about after you kind of figure out what you're doing. It's like, this could only be named if we, if we do it properly. But, you know, I've just made a note that, you know, I think the challenge of these systems of adding things like a synthesis or avail into an operation is to start trying to figure out how not to be yet another thing that somebody has to do. But, but the byproduct of, of the main things that you're doing and the workflows that you're doing should, as automatically as possible flow into these things. And, you know, I, I think about, you know, when you show synthesis, and I don't see how, if you're a large firm that you don't have that as your, you know, think of as like, that's your newspaper for the, that's your, that's your news source to keep everybody contextually aware of what is going on. And as a firm scales, there's just no way that everybody can kind of understand what everybody in the firm is doing. So it's a very, to me, it's a very elegant way. And then the trick, like you said, is it can't become somebody's job to go do all of those things because it falls apart pretty quickly. So it's got to be a, a byproduct of the normal workflows. And I know that's what we are always kind of trying to do to figure out, I don't want to give you another job to do. I want to, I want you to go do what you normally do. And then we'll try to, you know, be a byproduct of that.

Totally. I do think, though, like, yeah, as a good example, like, I showed that project management guide, like the folks that put those things together, and I know you've Both worked with firms that have probably done things like that. Like those are labors of love done by a couple key individuals in the organization. I don't think they were generally happy about making 100 page long PDFs with them because they're very hard to edit. Then you have to redistribute them. Like it's a terrible like authoring platform for something like that. So.

And it's a dead document, right? It's like, it's like it's one of those things where now I have to keep this thing up and I have to republish it all the time. And so like that the more Wikipedia style knowledge base that is constantly updated with multiple authors who can do some accountability with each other and they can hand it off and make, you know, if they don't have, they're working on a project somebody else can come in and do it makes so much more sense, right, Just to keep the life in these documents because they are constantly.

Evolving and you can put short training videos in the middle of these, which is harder to do with a document and you can add links to people and other documents. So yeah, it's a much better experience. But to Randall's point, like this is in the. Somebody's job is doing this like. And so hopefully we're giving them a better way to do it. So that's kind of if I can share my screen. So I, I have shown you kind of where synthesis is today and what I want to be talking about is where we're kind of going. And so our mission, like, kind of the way we'll describe the product in the future is around being an integrated intranet, LMS and enterprise search platform for AEC firms. So I touched on the intranet piece. I want to touch on Synthesis lms. I've just got three slides on it because again, it'll get a sense of how the intranet, LMS and enterprise search will all come together and the kind of content that will live in synthesis in the future. And kind of like what we're building for. So this is, and this is a mock, right? We're in design now, we're going to start construction this summer on this project. But this is a course in synthesis LMS called Project Communications. And I'm in a lesson called Client Communication Strategies. And in this example there's six lessons on the right hand side. You can see that I'm through one lesson. I can see who the instructors are on the right. And it's a, you know, an 18 minute video plus a handout right on Active listening Fundamentals. So we have seen our clients over the years try and use the intranet platform to do this kind of stuff and do reasonably well. And we kind of call it LMS Light. And we had just come to the point after finishing Synthesis 6, where we went out to our clients and had a big listening tour and they're like, we want you to work on two things, AI and lms. So this is the LMS piece and we're going to talk about the AI piece. All those courses can get bundled up into a course catalog. So in the example I'm showing you here, there's courses around onboarding and project management. But you know, our clients create courses around things like design, technology, sustainability, leadership, design itself, marketing and business development and so on. So this is meant to be one central place people can have all of their educational resources.

Chris, do you, do you, are you hosting those videos or do you Support Vimeo and YouTube and other places that are, you know, traditional places that people would post video content?

Yes to both. So we have a feature called Synthesis Native Video we released in 2018. We are in the process of doing a major overhaul. We're actually talking about that toward the end of the conversation around us using AI to generate AEC specific transcriptions for those videos. We're also adding things like chapters and some other neat features. But the Synthesis Native Video is a key part of kind of how all of this comes together. And we're going to talk about that in some detail. But we also do, you know, through embed codes, you know, you can drop in Vimeo, YouTube, that kind of something. And the last piece about the learning center is the ability to bundle those courses up into what we call learning paths. So for example, if you look under onboarding, there's two learning paths on onboarding. There's one that's called general onboarding, that's got like seven courses and then there's another one for project manager onboarding, which got, has got 10 courses. So you take that kind of core seven courses for, you know, everyone that comes and joins the firm and then you add three more to like onboard project managers. So the way we do project management here at the company, so you know, it will also have assignments and transcriptions and analytics and all the kind of basic things you'd expect to find in an lms. And this is scheduled to go into beta next year in 2025. So we have a 30 minute concept video laying all this out on the Roadmap page on our Website, maybe we can drop it in the show notes, but that's as far as I wanted to take it for what we're doing today. So let's talk about AI. So for us it's three things that all work together. There's next generation search, which we'll start with. There's video captions, which Randall, you just kind of teed us up for. And then we have a program that we introduced called Community AI which is going to help us build AEC specific models to make this work even better. So now that we've kind of seen the content, I want to do a deep dive into how we're going to make it even more searchable. So as we saw in that quick search example today, we're using keyword search. We've done pretty well with keyword search, but we've taken it about as far as we think we can. So I'm searching on keyword like jury duty. And maybe what I really wanted to do was ask a question, right? How many hours does our company provide for jury duty service? Or what's our jury duty policy? And when I execute that search, what I get back are links in this example to pages and to documents, which, you know, that's how we've generally done search. Where we're going is, you know, using generative AI and other kind of advances in search technology to make answers that summarize useful informations from multiple different sources automatically. So for example, for this question about what's our jury duty policy, we're giving part of an answer around allocation of hours. So how many hours do we give documentation? So how do you kind of like prove you were on jury duty? And then how do I request time off, the ability to ask follow up questions. And then if I drill in to one of these search summaries, I can link to that source, you know, that we. And we're citing the source where we got that information. So another example, I'm searching on work sharing and what I really want to ask is should I use revit, work sharing or central files? And then I want to get an answer back, right? So it's summarizing the benefits of work sharing, the benefits of central files, and then kind of giving a recommendation on the right, you can see the top sources that were used in kind of generating that search summary. And again I can drill in and find out the underlying information that was used to generate this answer. And if I click on it, that will take me directly into that video and directly into the part of the video that contains the passage that was used to generate that answer. So this is the synthesis native video. Randall, to your earlier question, that's how we're making this work.

So that's interesting because you're doing things that I think YouTube is trying to do as well. Right. Where they're automatically kind of looking, they're transcribing, they're looking at the conversation, they're. I know whenever I upload a YouTube video, it's like, do you want to automatically have it find moments or whatever they call it? They don't even call it chapters. Like you can put in your own chapters in the description field to help people navigate to a certain thing that I think you might want to navigate to. But they're also looking at finding smarter ways to find those key moments in videos. But then you're, you're locking kind of a timestamp and what was said to a question which makes it much easier to. Like I was talking about earlier, no one wants to watch the hour long video. Too long. Didn't watch. Right. I just want the key takeaways or I want to jump to a specific point in that video that is regarding the question that I have. So that's what you guys are really taking on here with your own video platform. Because you can then process that in the background and then apply all of that metadata back into the. The LLM.

Correct. And I think the other reason this is a huge unlock is, you know, because we work 100% AEC. And I'm going to touch on this a little bit later. There's so much AEC terminology and lexicon acronyms and product names and material names and all the stuff that we do in this industry. So much lexicon stuff.

So that's where YouTube falls down.

Exactly right.

Because it says rabbit instead of revit. And there's all, you know, that's just one example. But there's so many acronyms that there's so much terminology and jargon and vocabulary that it's great when you're in an office, you need to be using that because you're all working in the same field and you're all doing the same things and you have to be able to then like you're saying it's a huge advantage to have that vocabulary be a part of the system so that it finds exactly what you're looking for and it's not coding it as a misinterpretation. And now I can't even find it because I didn't search for rabbit, I searched for revit.

Right. Or I searched on standard and we use guide so much stuff like that. And so you're teeing me up really, really well actually. So that's it for, I've got for like of the front end and this project I've been talking about as an iceberg all along. Hopefully not in the Titanic sense of the word, but in the 10% of this project is UI above the waterline. 90% of what we're doing to bring that search stuff to life is under the waterline. It's invisible. It's infrastructure behind the scenes. And what I want to share with you and your community is kind of what's below the waterline and like how the generative kind of search results actually work.

Real quick, before, before you jump into that, I just want to see if you guys have this experience which is okay, so we've, we've all grown up with the Internet, right. And so we've all been trained how to use the search tools that are on the Internet to our advantage, which was typically keyword based. Right. So I'm a keyword searcher and it is such a mental leap for me to ask a computer a well formulated question. It's like you go into the ChatGPT, you go into Claude, you go into what? And, and it's funny because my, my wife will, she, she adjusted and adopted this way of doing a search a while ago which is just ask it a question and actually put a question mark at the end of it in the Google search. I still don't, I still can't even, I can't bring myself to do that because it's not my muscle memory. And so I'm wondering like with firms and, and how you're creating an integration with that because I would assume that if I don't even know the right question to ask, but I just know, okay, work sharing. As to your example, I can type in work sharing. And then maybe there's, maybe there's even other questions that have been, that have come up from other people about work sharing. So it's like, oh yeah, that's, that's really what I want to ask. Because me, I'm just a work sharing. I would say revit work sharing. Right, totally. And then see what comes back. I don't ask it the question. And now this is very much a communication, this is a conversation with the computer at this point. And I'm not conversational with computers. That's just not how I'm wired.

Yeah, I think you're on to. And I Think you're right for probably representing most users. And so I think for us we have to do really well when people are still using keywords in synthesis search. And I think for example, let's take something really prosaic like WI fi. Like if somebody's asking if someone types in WI fi, what are they asking? They're not asking which WI FI technology do we use? They're not asking like do we have WI Fi? They're asking what's the, you know the WI FI network in my office, what's the password? That's what they want to know. Right. So every firm meeting, every conference room.

I go into, they cannot get me on the wifi.

It takes five minutes every time. It's a little dance. Yeah. And so from a meeting people where they are perspective but also from just an efficiency like why type 10 words if I can should be able to get what I need with one word. And you know, the more people type the more the typos spike. You know all the, I think to.

To your point Evan, you know you can, you can begin as you're capturing those interrogations questions that are being asked. You can begin to make those suggestions as somebody starts to type something in like hey, these are the popular things that have been asked before even for the user to not have to retype it, right?

Like okay, so useful, right? It's like autocomplete for what I was thinking. Not even what I was typ.

Google's gotten really good, right? It's like content, you know, it's contextual awareness and then just start to type something. It's like knows what I'm getting ready to.

They've got a nice big user base.

Pretty big to pull from. I mean I think that's one. The tool that I've been super inspired by is Perplexity. I don't know if either of you have used that yet, but it's fantastic. And it like really handles that from one word all the way through a complete, you know, multi sentence question. Well and so it really depends and sometimes like one word does it or two words do whatever it works. Sharing would be fine. Or if it's like yeah, I see what you answered but like really now I know what it is I'm asking. What I'm asking is you know, xyz.

So that's usually where it leads me, right? It's like okay now okay, now that I've seen a little bit more I can ask a little bit better of a question. And like five questions from now I'll actually get 100%. All right, back to your iceberg.

Back to the iceberg. So when we, when we started approaching, you know, obviously when everything changed with ChatGPT, we started asking the question, like, every tech company. Well, every company writ large, not just tech companies. We're asking. It's like, okay, what does this mean for us? What does that mean for our clients? And we quickly kind of keyed in on search and video transcription as being two important things we wanted to build. They weren't little quick hits. You know, we didn't rush something out, but we really took a lot of time thinking about architecture. And so one of the ways we could have built this was to build an end user facing large language model, right? And so questions would come into synthesis, we'd use a LLM that we had trained, and we'd generate answers, right? This is basically. It's overly simplified, but this is basically what ChatGPT does. And for a host of reasons, like this was the wrong architecture for us. The challenges are that it is prone to hallucinations just because of its data set will oftentimes, you know, that goes across, you know, different companies. It can be a really, a really big problem. It's got a problem with freshness because, you know, you train this model and then you, you know, whatever, train it nine months or 12 months later. And so it starts getting a gap in terms of stuff that's either you're holding on to old information too long or you're not recognizing new stuff fast enough. And then it's got a permission issue, like inside a company, it's really hard. Like, if we have, for example, principals or a shareholders or a management team community in synthesis, like building an end user facing LLM, like, who can see what. What's used to generate. It's just too much. It's too complicated. You know, it's not what we like the sound of. So it's like, all right, is there another way? And what we came across is this emerging approach called rag, which stands for Retrieval Augmented Generation. And we love it. And this is kind of how it works in a nutshell. So a query comes in, like, what's our GEODD policy? We looked at this earlier. We execute a search, and so we execute a vector search. And I'm gonna come back to that in a little bit. The vector search retrieves passages of key text from those relevant resources. So out of videos, the right passage out of a document, you know, it's on page 20. The right passage out of a post or A, you know, a page in the platform sends those passages to the large language model which will then summarize it and generate those answers and citations we looked at. So we're not kind of indexing like all of the knowledge to make an oracle that knows everything. We're using search and then we're just using the LLM as a summarization tool because it's very good at doing that.

Yeah. So is it, would you, would you call it more of a just in time kind of a Q A machine or I mean how it works?

Yeah, I think that's great. I mean I think I, if I put a post up right now, it gets, by the time that it's done indexing and it gets embedded, we'll talk about that a little bit later. Then the next time I do a search one minute from now. And it's something that was related to that post I just added that would be included in that search and fed into those search summary. So it very much is a.

This goes back to that dead document idea and your freshness concept that you, you brought up a minute ago.

So it was super important to get something that really tackled, you know. Well, let's talk about it. That really tackled hallucinations, freshness and permissions. So you know, we counter hallucinations by grounding the search in your data. So this only searches, you know, your company's data. It doesn't do anything across, you know, our different clients or from the Internet. Right. This is really your content. So again that's that search that retrieves the passages, passes it to the LLM and then generates the citations. The real time search, you know, you just mentioned that's a big part of it. And then permission search. So if Randall is the principal and Evan, you know, you're a job captain, you're going to get different passages retrieved based on your permission level that get fed into the LLM. So we can do this summarization but we can do it in a way that honors permissions.

So when people, how are the permissions handled? Are they handled during like the upload of the new information or is it based on like places, I'll generically call it places in the Internet. So if it's like hr, hr. Everyone in HR has access to everything in hr but outside of that there's like certain eyes only. Right. So how do you handle that when new information is constantly being uploaded because are being added to that still going into kind of documents and then like there's permission for that document in that folder that the LLM is in looking at when you ask it a question or how do you handle that?

Yeah, it's very much the latter. It's based on places we call them communities and we have public communities and private communities in synthesis. And so public communities are what they sound like. Anyone can see anything in that. Within a public community you have different permissions for who can edit content versus that are end users. But everyone can see that content. And in a private community you have to add, you know, users to it manually. So that kind of. But that's as far as we took it. You know, like SharePoint as you two may know. Like you get so granular the permissions, you're down at the document level. It makes things super complicated and people can't remember who can see what. So we launched Synthesis 6 with a very simple permission model and it's stood the last couple years of testing really well. So that's simplicity is always good if you can get it. So that's kind of the basics of retrieval, augmented generation and you know, how it improves search. So there's kind of three other concepts that are important to how we're building vector search, which I lightly referenced earlier, our community AI program where we're building those AEC specific models and our kind of our approach to enterprise search. So vector search is super interesting and I'm going to overly simplify it and if there, I'm sure there are some people in your audience, in your community who know about vector search and they're going to be like, dude, you're totally oversimplifying. It's like, yes, I have acknowledged I am oversimplifying this. So perfect.

Perfect for me.

Thank you. So think of it as like a multi dimensional space with thousands of possible vectors, right? And so basically what we're doing with vector search is we're turning text into numbers and coordinates really. So for example, programming might be near floor plan or space planning in one sense of the word. Programming may be near coding or software development kind of in another sense of the word. And programming might be near scheduling or event planning in yet another sense of the word. So let's go back to rag to see this in action. So when that query came in this time around, do we have any healthcare programming templates? We execute a vector search which goes out to a vector database. So it's stored all of the locations of all of our content using those coordinates. It then retrieves the passages of the key text from the most relevant resources, sends them to the large language model for summarization, and then it can do the citations. So the question is, how does the vector database know which passages to retrieve? And the answer to that is getting into our vector search ingestion pipeline, which is exciting as it sounds. So whenever anybody uploads or edits a synthesis resource, that resource gets broken down into smaller chunks of text. So, for example, let's take a synthesis page, and this is a simple one on time off. So we're going to break that down using the page headers into chunks. So we've got a chunk around bereavement time, a chunk around jury and witness duty, and a chunk around pto. Or this is a feature in synthesis I didn't show. But these are collapsible sections, right? So we've got all these different collapsible sections having to do with payroll, one on W2, one on how our raises and bonuses decided, et cetera. So we use those as signal on where to break up a page, and then we put them into those smaller chunks. So we take those chunks and we send them to what's called an embeddings model. And I'm going to go into this in a little bit of detail, but the embeddings model knows how to look at those chunks of text and that it embeds coordinates, it embeds vectors into them based on their meaning, and they look like this. It's really just a string of numbers. It's complete nonsense. But the vector database and the embeddings model know what it means. And so we add those chunks in the right location using that vector. Right? And so this will come a little bit clearer at the next step. But the key to this whole organization is the embeddings model, because it establishes the vector space in which meaning is assigned. And because this is such a key. Am I good? Should I pause or should I do. Are there any questions you guys want to ask or should I keep.

I have a. I have a question, because video in the example that, that I know we'll be talking a little bit about, but it's just a run on conversation and, and so there aren't these chunks that you're talking about necessarily. And I know, like, I think this goes back to like the thing that we were talking about earlier with YouTube kind of trying to identify key moments. And so maybe there's sentiment changes, maybe there's. There's pauses. I don't know how it's doing it. Maybe you have more insight into that. So my first question is there, it's like when the document does not have any kind of hierarchy to it, there's no structure. This is what AI seems to be pretty good at. Right. You can throw like a giant thing and it can kind of figure that out. Like, I use this all the time. I use Notion and I use Notion's AI feature and obviously ChatGPT too. You can summarize documents, you can say, tell me the three key point, key takeaways from this conversation. And it does a pretty good job at that. And so I'm just wondering, is that, is that really helping you here as well? And is that, is that really how you're attacking this with.

I think so. The reason I say I think so is we're evaluating a couple different approaches and we don't know today, on April 25th, which one we're going to pick, but that is certainly one of them is to. And you know, it doesn't have to be, it doesn't have to be 100% precise. It just has to be good enough, you know, to break it down like this. We're going to be able to find, you know, what we need anyway. But I think your point is, right, that's. And that is you can read. I've actually read a lot about how the Google key moments things works. And it's a lot of what you're saying, it's like looking for changes in inflection, it's looking for breaks. It in some cases it will even OCR the slides, if there's slides, or the video, and kind of get a sense that something's changing there. So yeah, there's a lot that goes into it. But it's super interesting how they built that. So yeah, that's when we look at like unstructured like a video, which is as you said, just like this long rambling thing we're gonna have to infer, put some structure onto it versus just like break up every 200 words like it. That's probably too blunt.

That is where you have. The context becomes everything. Right. The better, the better context you can put around all its information, the more accurate your results are in.

Yep.

And I think, you know, to the question or to the topic about, I mean, you're using a great example here with programming, right. I think of architect programming. Clients don't even know what that means, but we use that word in front of them and they, they learn what it means going through the process. But if you were to bring up programming, they would never guess that it's about spaces and adjacency. And size and scope and all those things and square footage, like they wouldn't, they wouldn't think about that. And my question is, is the older generations in our firm who are the ones with a lot of that encapsulated knowledge, like how are you getting that out of them? Because a lot of this just happens through conversation. It's not recorded, it's. And so I mean, do you have advice for firms on how to capture this information moving forward to get that put into these models so that these models are smarter? Because again like a lot of this is just spoken, it's never recorded in, or maybe it's in an email here or there. And trying to like dig all that up and get it in there to try train this from my firm would be really hard. I mean, are you just doing that on behalf of everybody with your opt in firms that are partnering with you to help train this AEC specific model?

I mean it's a great segue, right? So we are offering two different embeddings models to our clients. So for the folks who opt out, which is the default, they'll just use a generic open source embeddings model. And for the people who open, who opt in and contribute content, they'll use the AEC specific model. But I want to kind of go back to your question a little bit because I think it's a super important one. It's one of the reasons we're building an lms. I mean we're obviously primary reason we're building an LMS is to help our clients grow and develop their people. And it turns out that, you know, what's really important isn't just volume of content for what we're trying to do. In the kind of search case I showed, we want high quality content and we want a lot of it, but we want the quality part's really important. And so when people generally take the time to put together and you know, this could just be a simple hour recorded zoom on zero carbon buildings or something like that, or like how to work with healthcare and like here are the main phases and like we're talking about programming is one of them or something like that. What I have observed in my career and while watching our clients is that, you know, the people that put those things together put a lot of thought and care into them and then there are interesting discussions that kind of come out in the Q and A. And so it's why video is such a core technology for us because it's easier for them to do that. You know, Zoom meeting and kind of share their knowledge than sit down and write a guide or write a document or write a long post. And then if they can, and this is why the LMS now it's like, okay, but then we can reuse that content. It's not just the lunch and learn that happened on a Tuesday in March. This is now a course that can be for future employees that come along that want to learn about, you know, net zero projects. And it can be assigned or it can be recommended depending on your career track or your learning path. And so we get more ROI out of those videos in terms of educating folks. But then we get more high quality content we can use to answer people's questions but then also train these models to understand that aec.

I wish, I wish both of you guys had been able to be in New York with us last week for the Confluence event that we had. We had two of our speakers were from Thornton, the core studio with Thornton Tomasetti. And it actually wasn't part of their presentation. But afterwards, as part of the discussion there was a gentleman that worked at Thornton Thomasetti for 30 plus years who passed away recently and they built an AI engine. He had documented like, I think they threw out the number like 10,000 interactions that were through either through email correspondence or Q A. So he was literally the, you know, the, the, the person that knew the most. You know, it's a really beautiful thing when you think about it as, as, you know, how do you capture that knowledge and experience in such a way that can give to the next generation? Because it's, you know, and I think the, you know, what you're working on. Chris, the, the, this kind of interaction with that kind of information is how, how can that legacy and all those learnings pass on to the next gen? And we don't lose. Yep, we don't lose what was valuable. And we got into the discussion about, you know, in the end when you get a lot of information, it's like what might have been true 20 years ago May not be true today. So you have to fight that part of it. The other piece I'll throw out and I'd love to engage with your in this kind of thinking, but I've been throwing out, it's like a lot of the work that everybody's doing on AI is, you know, we have a lot of information in this industry, especially in the form of kind of, I'll just call it finished product. Like here's what we designed, here's what this ended up. Right. So, so you can easily go and look at that. What, what seems to be missing is the how you got there. It's the, it's the why. Why did you make that decision?

It's the stories. Right?

So I've been like, yeah, kind of throwing out this like what you really want to start happening is and I actually past weekend post that confluence event, I actually went on Amazon looking for. I want a red orb. I want this like device that when we're back in person with each other I want to like drop that on the table and hit record. And it just be this, it's just gathering the conversation and the why as part of the process because I feel like that's part of, you know, we're gonna, we're gonna get the end result of this stuff, the polished end result, but we're missing the explanations of that. So anyway, I'd love to hear kind of what your thoughts about where this, it's going to end up. Where does this go?

Yeah, I'll just, I'll give you so the in, in. In. I'll give you some KM speak for what we're describing which is critical knowledge transfer. And so critical knowledge transfer talks about, you know, especially when you're talking about. In many cases it's used for senior people who were there at key moments, founding, launching new services or new market types. But it's also when you've got, you know, a super rare subject matter expert and you want to just be able to like leverage their knowledge to other people. We, we ran an entire day at K Connect, our annual conference a few years ago on critical knowledge transfer. We brought in an expert. She was a former professor at MIT and Harvard in the business school. This was her career. She wrote a book called Deep Smarts which I highly recommend. Her name's Dorothy Leonard and we ran a project with four of our clients over the course of six to nine months where Dorothy kind of like used our critical knowledge method methodology. And we just ran projects in those companies to kind of pick out a couple of those experts and figure out how to flesh this deep why stuff out. Like it's the, it's the deep smarts. It's not like you know, simple, simple things like the project details and the square footage and whatever. It's like wisdom. It's wisdom. There was a story from Dewberry that was amazing. There was this very senior long tenured engineer and the thing that came out that he had that just people didn't understand is he had this list of seven priorities. When you're working on a project. And what he said is the important thing is the order, right? Because that's where values are determined. And so it's like, you know, for example, safety, like it doesn't matter, cost, scope, all these different things. It's like if this thing is unsafe, we can't do it. And so it's kind of like that's a deep way of thinking about how to design something. It's a framework. You're right. You want to excavate these frameworks that a lot of times these experts don't even know that they have, right? They're implicit and then they're in the back of their brain, but they don't know that they use them. And so that's what this kind of process of critical knowledge transfer is about. It's about excavating those things. And they're oftentimes done through project stories is how you get them out of people, right? You say like, okay, well you did this on this project. What did you consider? What alternatives did you look at? How did you make the decision that you did? And through that process of giving those exact examples, that's where you start finding out those kind of hidden frameworks that people are using and their heuristics that they're not even necessarily aware of.

I think about, you know, especially I'm a great example. I'm not right out of school, but I never practice. So it's like I've got, you know, went to architecture school, but I never practiced. So I'm, you know, I don't have all of that deep, you know, knowledge, practical knowledge about some of this kind of get it in theory. But. And I, I've used this example. I have a good buddy of mine that I went to architecture school with who now owns a firm practicing and I had this, I had the PDF with details on it. To me they're just detailed. As soon as I showed it to him, he starts pick, you know, picking at them and explaining different parts of it. And I've used that as the example of like, you know, if I, if, if I was the. I'm proverbial, I'm, I'm like perpetually a 24 year old in my knowledge of architecture because I never did really progress from the, you know, I understand conceptually what it is, but I don't know, I'm not intimate with that. But what we're actually building some tools, we're building some annotation tools in avail and I'm very. Right now it's kind of redlining and the traditional. But I'm very keen on starting to capture video and audio part of the annotation process. Because what I'd really like, when you're marking it up, if you'll just record that, explaining at. In the moment why you're doing it. To me, it seems like that's. We've never really had that. So I'm kind of keen on starting to experiment with will people be willing to just record an audio. And I think, Chris, the work you're doing is the right way to think about this. That people, they don't have to go type it up. I'm not going to go write a blog post about it. But while I'm doing this, I'll talk. And that's the easiest form of me getting this information out. And then it can be chewed on and perpetually available, you know, in perpetuity.

It's like, no, you, you. So I mentioned there were four firms as part of this critical knowledge project. One of the other ones was Shepley Bulfinch and Jim Martin and his team did a couple experiments. One of them was with kind of the senior, you know, quality ca kind of person who was reviewing sets. And they recorded him with a junior person reviewing sets. And as he's redlining the role of the junior person is saying like, well, what did you see there? Why did you circle that? You're like, why did you make that note? Because the expert doesn't necessarily know. It's just so intuitive. Like it's tacit knowledge at this point.

Right?

And they don't say.

Yeah, I just think you should know it. So that was one of the experiments they did. And they did this other cool one with one of their healthcare experts where they would show a picture of a, like an operating room or something like that. And they would ask her to say, like, what's wrong? You know, like, what's broken here? You know, and she'd say like, well, that thing's not decode. That thing's not decode. I'm looking at that. Someone's going to trip on it. And like, just like the knowledge of how she can just process one simple photo and just like tear it apart. Like, that turned out to be super useful. And so they like turned that into some. That's how you start excavating some of those things. So those are a couple of good examples.

Exposing that pattern recognition into verbal and video, like, is a huge deal. And I was just going to say, Randall, about like the markup and talking out loud. This is Actually something that you could do in a zoom meeting. And you don't even have to have somebody else in the zoom meeting. Right. You, because you have a whiteboard, you can load up a PDF and you can record it to the cloud at the same time. And now with the Zoom, I think they're using Otter's engine to do transcriptions. And it just seems like a great way to create. Start creating a library of captures that are video plus markup plus transcription takeaways, all those things that you could potentially use to train your staff in the future because of models like that.

This is all going to be making it back as, as part of the normal workflow, not be something that you go do afterwards because that is a behavior change. But maybe it's not too difficult. Like, hey, just verbalize what you're thinking.

Yeah.

And say it all. Outgrade.

I think it, I agree. I think it's not hard because I think, you know, in, in kind of, you know, mindset is the most important thing for behavior change. And I do believe in general, in this industry, we have a culture of mentorship and wanting to advance the next generation. And so I think this is just a different technique, but it fits into a through line of something people already want to do, which is altruistic. Yeah, it's a very, it's very altruistic. It's a great profession. Let's talk about altruism. That's a great segue to community AI. So let me tell you a little bit about what we did. So when we saw this opportunity to build these AEC specific models, we're like, how are we going to do this? It has to be opt in. You know, we're not just going to start scraping everyone's data. But we saw that there was this great kind of quid pro quo. Right. If you help share some data to help us train these models for other folks in our community, you'll be able to benefit them, benefit from them. So the two models we're building are the embeddings model, which we just talked about a little bit for more relevant search results, and the transcription model for more accurate video captions. And so our clients are opted out by default. Once they join it, they get access to the model. So at a high level, we're extracting what we're after, our AEC specific terms, phrases, and then the context for those terms and phrases from content at the firms that participate. So, you know, here's a word soup example. You know, anything from acronyms to products to different Kind of systems like these are the things that generally the embeddings model out of the box, the generic one and a generic transcription, video transcription model don't do that well. So that's what we use to fine tune both the embeddings model and the transcription model at a high level. What we're really trying to do is kind of build an AEC crowdsource dictionary from our clients, right. So that we can do more smart things with AI.

Makes sense regionally even people are going to use different words for the same thing or even in the same firm, right? Different people are going to call gypsum board or drywall or gypsum wall board or there's going to be an abbreviation for that which ties back into content management, like what Randall's tackling, right?

It's like you would abbreviate it on.

A detail differently than you would even say it out loud. And so all of these things have to tie together.

Tie together. And that's what's the beauty of a vector search versus keyword back to the original piece, right? Because vector, it's about relative proximity, not exact match. Right? So these terms are in a neighborhood, right? These terms are all kind of in a neighborhood together. And like these terms are all kind of in a neighborhood together. They're not exactly synonymous, but they're close. And like that's how we start understanding the relationship between different ways of saying the same thing. The last thing I want to say on the search front, and if we still have time maybe good to go into training the video transcription model a little bit is our vision for the product is to be able to answer questions using your highest quality source of firmwide knowledge, whether that lives in synthesis or not. And so we do have a history of doing this. There's obviously synthesis content and the synthesis LMS content when it's released. We've got integrations with ERP and CRM systems. From our directories and profiles that we saw, we integrate with Open Asset for imagery and new Forma for contacts and we do Zendesk for help Center. So that's the. This is all stuff we do today. So if you were to search synthesis today on work sharing, you can see that one of the sources we pulled in was from a Zendesk Help center article on Revit work sharing. So we just have pretty ambitious plans to expand and make more sources of that firm wide knowledge available. So that might be something like teens like looking at kind of messages and documents from public channels within teams that might be Something like freshworks, which is a Zendesk competitor. And it's like outside of Zendesk. It's the most common help center software we're seeing in our community. We're likely going to build a search connector API that will allow clients to build their own integrations to push content into synthesis search. And we're looking at this idea of a public website indexer. So we've kind of gotten two use cases for this. One is to index their own website. It's very common that clients will do high quality content on their website that doesn't make its way into synthesis. And then in the second use case, it's to index the help centers of trusted software partners. So maybe indexing the Avail help center or building some kind of integration with the Avail help center or the Open Asset help Center or the Autodesk Help center or whatever it might be. So these are things we're exploring. We're definitely committed to building the search piece I showed you and then the teams and the fresh work and the search connector API and public website, these are all future explorations that we're trying to validate and see if this is worth building. So the, the idea for that is this is going to go into public beta in 2024, so by the end of the year.

That's great. Are you thinking, Chris? Because we're kind of, because we're experimenting with a bunch of these different kind of search methodologies and changes in search methodology. Do you envision, like with synthesis, are you all working through that? This can be a single search box, like the Google search box that you'll begin everything from that or do you have to like quickly, quickly put people in proper directions to get to the right info? Because if you do connect all of those things you're talking about, it can get noisy again, right?

Yeah, yeah. It's a good, I mean it is a very good question and I don't know all the answers yet. I'll just tell you a couple things we're thinking about. So on one level, look, there were some systems that weren't mentioned there, like email, for example, or file shares. What, what we think is definitely important is kind of blessed, stable, finished, high quality, evergreen knowledge, you know, so the kind of stuff we find in synthesis or in a help center, you know, that is like kind of blessed. It's for everybody. And this is kind of, you know, when we look at email, that's like stuff that's in flight, you know, teams, like a team. I think teams is definitely like it probably wouldn't be everything in teams, but it's noisy. Yeah. It moves in one direction. It's a fire hose. It's a lot of jokes and asides and half ventured opinions, which is great. We need all that stuff. I don't know if that's what you want to put into a firm wide search. So we're looking at every data source to kind of say like I'm looking at what Microsoft's doing with Copilot, right. And what they're primarily overlaying is like teams and email and file shares and not as much kind of like firm wide, you know, sources of truth. And that gives you a very different thing, you know, like, or I think of someone like a new forma searching across email. Like there's just a lot of email in there and, and what they do is very valuable and what Tonic DM does is very valuable. In terms of email search, I don't know that that's the kind of content we want like in a synthesis knowledge based search. But to be determined, you know, this is early in, we're all kind of.

You know, up against that. Like there's a lot of different sources, you know, but even you know, in Google, you know, I use all the time like, like Google Drive for instance, you know, if I know what kind of document type specifically that's, that's probably my most clicked thing. I know it's a spreadsheet, I know it's a right a word doc. So then you know, that least contextually, you know, weed out lots of noise. So you know, I don't know, there's just challenges with the UI and UX when you start to have, when you bring back a lot of search results and it can be very noisy. It's like, okay, what's the second level filtering? And, and then there's always the question.

Of do you totally.

Do you preset it or do you post set it?

Right, like, exactly. No, I think so. So this is the second point I want to make. You took me right to it. So I think we're going to end up with something like, you know, I mean an example, everybody would know. It's like on Amazon, you know, you kind of search everything or you search books or you search prime video or whatever it is. Perplexity has an interesting feature called Focus where it can kind of, you can say, well I just want to search Reddit, right? Or I just want to search YouTube videos because I just want to limit my search. And in the synthesis application that could be, look, we've got a library of all of our proposals for the last three years. I just want to execute this search against those proposals because I don't want other noise kind of getting in here or I just want to search against the lms. So I do think it's probably a mix of what you said, Randall, both pre filtering and post filtering. It's probably a mix of the company will probably set up some filters in advance, but then you personally probably can go in and set up some different filters. So I can go in and do I'm doing my proposal search or I'm doing my knowledge base search.

I found it interesting, you know, different people think differently. Some people want to pre do things and some, you know, we've taken the approach of, you know, tell us what you want, we're going to go gather up all the results and then our job is to give you enough context to, to, to make the next two clicks, you know, narrow that down. There are, there are other solutions that are out there that are, you know, you end up with a more complicated interface because if you want to pre select all this stuff now, now I'm presenting all this stuff and I'm asking you to go make a bunch of choices and then execute the search. So we've kind of taken the former approach. Yeah, but I think it's, you know, I'll give a slightly different example but like we've, because we've got a tight revit integration. One of the things we've done with avail is, you know, you can find a piece of content like a piece of revit of revit family and it's got type catalog. Well, what most firms want, BIM managers want is I want the user to only bring in the type of that they're going to use. Because I don't want to like preload all of this data into the model, into the BIM model if it's not being used rather than you're just bringing in. So we've kind of designed this approach that is, hey, when you, when you bring the family in, we're going to tell you the types and we want to right there for you to pick that. It's interesting though because people will count the number of clicks it is and say, oh, this is too complicated because it took four clicks to get that in as opposed to drag the entire family in. Then, then I got to go open the property type catalog. Then I've got to do this then because I've changed the width, I've got to go move. They don't count all those. They count, you know, only what they want to count on the front end of, not the post process. Totally true.

You know what's interesting about that? There's a. There's an article by Nielsen Norman Group, who is an awesome UX research organization and they've got an article, I think it's called like the myth of the three, the three click myth or something like that. And basically after doing a bunch of research, what they kind of established is people will click more than three times if they feel like they're making a reward.

If there's reward along with reward. Yeah, it's just. And I don't know what the answer is. I just know I get frustrated sometimes when I have people being like, yeah, you know, we didn't really like it because it took four clicks to get this in. I'm like, you forgot to go measure what you had to do after.

Right.

You were just considering what we're doing on this up front anyway. It's just different approaches and you know, I do think people have different mindsets about how they think about these kinds of things, so. And ultimately in a large firm you have to support in some, some manual, correct?

Yeah. Well, it's like browse versus search too, right? Like I showed you guys the mega menu stuff on synthesis. Like some people are going to browse the stuff and some people will never do that and they're only going to search the stuff.

So yeah, they'll just type something in. It's interesting also like the accrual of, or the accumulation of all of the extra assets and, and that is like a technical debt at some level. Right. It's like somebody clean that out at some point and they're not even thinking about all the clicks. Somebody else down the line, BIM manager or who, whoever else. Not, not my problem. It's somebody else problem later.

We call them assets. Maybe that's the wrong word. Right. It's built in.

It's built.

I've always, I've had this long standing debate like where, where is your content on your balance sheet? Is it a. And I said a. Reliability.

That's good. I like that. That's funny.

And then where are you in the project and who's doing what? Because as a designer early on a project, I don't even, I can't be that specific yet. So I'm going to throw a bunch of stuff in there and I'm going to, I'm going to slowly narrow it on to what I want because things are in flux all the time. You know, somebody says something in an hour and it's going to change what I just did. And so, I mean, that is also a reality of the field that we're talking about, where it's just constantly probably are much better.

You know, I only know this industry because I've spent, you know, my whole career in it. But I get a sense that the amount and the breadth of information that has to be dealt with and managed in this profession is maybe, you know, order of magnitude more than most other professions. Is that true?

That seems right from. That seems right. From what I've seen, I mean.

Yeah, it feels right.

It seems right.

Yeah.

Yeah. Do you guys want to take a little talk about video transcription?

Before we do that, I just have one more thing I want to ask you, which is. Okay, so. Because I'm really interested in, like, the idea of AI agents and because everybody, I think is going to be t. You have, you have all these exterior partnerships that you're talking about or connections that you. You've got Deltec, and you can list off all the different ones. But like Open Asset, for example, okay, Open Asset is going to have their own AI thing, right? And so my question is, are you talking with them so that you don't have to duplicate any of their work in your stuff so that your AI agent can talk to that AI agent? Because I assume, like, they're going to be able to go much deeper into image search, for example, not just metadata that's attached to the post that that image is in, like the project and the location, all that stuff, but, like, what's in the image. Right. So I could, I could just see it being this AI agent thing. It's like, it's like, tell me what, what product was used in this image? And I can, I can just point at an OpenAI asset that I found in synthesis, for example, to. To get me my answer. Because I want to know, because I need that answer right now. I'm on a problem.

Yeah, I mean, I think agents will be. We're not doing it right this minute. So I. Is all conjecture, but I do. Yeah. Like, so, for example, Open Asset has the ability to automatically generate proposals and, you know, qual sheets and that kind of stuff. So that's something we're not going to build. And so it could be that, like, a user finds five projects and they're like, okay, these five projects, hey, Open Asset Agent, can you make me a proposal out of this or make me some kind of RFQ or a brochure? Or something like that out of it. So, yeah, I think agents will be part of our future. I think it's just not, it's not here today, but it's.

Yeah, you either.

So this is all, this is laying.

A lot of groundwork.

That's super interesting because then I don't have to go to all these different places to do all the different things and then assemble it. It's like, it's like it just goes for me.

An API that allows a third party to query for the important things that your platform does and ability to get that info back. There's probably also going to be the, you know, when you talk about these chatbots, you're talking about an interface behind that, the model. You're probably going to start to have a ability to just connect directly to exposed models that somebody may expose, you know, via some API or some connection to that. So you may train a model and then expose that. Other bots can have access to that model. But then, you know, back to the question I was asking earlier, Chris, like at some point you have an interface in a box, but it's going to need to talk to potentially thousands of different, you know, hundreds of different sources of where this information is coming from.

Yeah, I mean, the iceberg gets bigger. Yeah, it's going to be a challenge where the interface is still the simplest piece of the whole thing. Yeah, and exactly. And so the stuff we're putting down now, the infrastructure we're building today around vector search and embeddings and all that kind of stuff is totally going to help us get to that future that we're talking about. Like the UI I showed you was much more of a search ui. Does synthesis end up with a chatbot ui? Maybe. Is there some other paradigm from a UI UX perspective that takes over in 18 months and replaces kind of the search and chat? It's very possible. And so whatever happens on that kind of UI ux, you know, advancements like that core under the water line stuff is going to be important. And I don't want to say it's just UI in quotes, but like to some degree that's true, that that's going to be the easiest part to move around. It's that complexity of the integration, the understanding, meaning coordination with other AI agents, like all of that stuff behind the scenes is going to be really the different, in my opinion. It's going to be.

Yeah, there's also, you know, with the, you know, vector databases and the RAG models and, you know, they're very good at what they are being used for, but they don't replace. Still some other valuable ways the data. Databases.

Databases, yeah.

Are migrate into one direction or another because they become super powerful at doing something. And so I think part of the challenge is going to be we're going to start mixing search methodologies, which means the data's probably stored in different kinds of databases behind the scenes. And then this is where the UI UX challenge comes. How does a user keep this?

I've got a lot to say about that. So. Yes, so you're right. So database is part of it. You know, we, the vector database is another part of it. Like we'll probably end up building a graph, you know, at some point. And so so much of the magic of making all this work is going to come down to understanding query intent. So for example, the, like some of the example, the WI FI example I said before the jury duty, like that's pretty clear. That's a, that's a, a knowledge based query. If someone asks a question in synthesis, how many healthcare projects have you done in the last five years in California? That is not a knowledge based query. That is more like a knowledge graph, database type query. So we have to really understand query intent and then execute the query differently. And we train that using machine learning to be able to know when it's this kind of query. Here's the way that you use your resources that you have access to, to answer it. So all of that is like, how.

Do you manage rhetorical questions, Chris? It's going to be hard. There's going to be times when people just say something out loud and then it's like, yeah, I really wasn't looking for the answer. And you can't guess there. I know, I could just imagine like humans are complicated. The way that we speak is there's so much going on that is below the surface. It's the iceberg is a good example.

Exciting is that there's always these little pivot points in the technology development. So when the new technologies come about, I know for me it's like it just reinvigorates you about, okay, you know, things that we thought about or were thinking about now kind of take on a new, new era of possibility because we can apply those technologies in some new ways. So I think it's exciting times to.

Be working on it 100%. I've been saying in the last six months so far, this is the peak of my career, I'm having the most fun now that I've ever had for that exact reason. It's a new wave of technology that lets you build on all the stuff you've done before and do new things. Super cool.

Well, let's talk about AI transcriptions of yeah.

Which is I think a much more interesting topic than it sounds like on the surface. And you know, it's a little bit like, well why, why are you going all in on captions? Like why is that such a thing? And I mentioned accessibility is really important for us. So it's part of an accessibility initiative, the LMS and the search stuff. So it seems like an important technology. So it's pretty simple in terms of the diagram for you know how we do this. Someone uploads a video. We're using Azure Cognitive Services speech to text. We have two different transcription models. We have the generic one which is the default. We have the AAC specific one which is through community AI. And then we generate captions and transcripts. And so again the opportunities to build something AEC specific. So I want to take you into the process of doing that because that is, it's an alpha now and the way we use alpha that means it's internally and we're about to take it into beta and I want to talk through kind of the steps of building that model. So step one is determining what we call the AEC terms of interest. Those are the terms we're going to end up caring about. We then want to acquire context for those terms of interest so we understand what those terms mean and how they're used in sentences. We want to fine tune a generic model with that AEC context. We then do alpha testing with benchmark videos. We go through beta testing with representative clients. We do deployment to all the people that are participating in community AI and then continuous improvement. The lion's share of what I have to say is really in steps one and two and then the rest of them are pretty self explanatory. So when it comes to determining AEC terms of interest, we have three ways that we're doing that. One is to identify AEC specific terms and by and I'll show you how we do that. The other is to identify common terms with AEC specific meanings. So we talked about programming before. Interview is another great example. Right. I don't know of any other industry. These interview to talk about going to try and get a job, they say pitch.

Going after.

Yeah, going after work they say a pitch or we got a meeting or presentation. Business development. So many things.

Yeah, right.

And then there's acronyms. So those three things kind of make up the AEC terms of interest. So we thought we were very clever and we were only going to have to do step A. And as we got into it we realized B and C were necessary. So this is what we're doing. In step A. We're extracting and de identifying lists of unigrams, bigrams and trigrams from synthesis context. And a unigram is a single word, for example Accelerate, access, acoustic. And a bigram is a two word pair like account access, action plan and acoustic analysis. And then a trigram is a three word pair. Annual revenue forecast, annual performance review, acoustic installation material. And so you'll see this is a mix of generic terms and AEC specific terms that we will pull out of synthesis content. And you can also see that we're starting with posts, document and pages. So we're obviously going to do video down the road. But like we need to do the video captioning before we can extract content from video. So the first step, it's a two, two step filtering process. So we're removing generic terms using these lists of common English unigrams, bigrams and trigrams that we've assembled. So what that gets you is, you know, I'll strike out in the unigrams up top I struck out accelerate and access. And now I've got acoustic abutment and otter desk. Like that's pretty good for a raw list. In the bigrams I've got ACME strategy. So ACME is meant to be a placeholder for a firm name. So ACME strategy, acoustic analysis and adaptive reuse. And then we've got acoustic installation material, air quality monitoring and anchor bolt installation. So that's our raw list. It's still not good enough yet. So we're going to do another filtering. So we are going to filter terms which don't appear at five companies or more. So we're applying this filter for two reasons. One is to help us kind of separate signal from noise and prevent us from overfitting the model on just one firm's terminology. But it also has this side benefit of alleviating concerns around confidentiality and intellectual property. So for example, that ACME strategy example that we saw in the first example wouldn't make it through this filter. And it just gets knocked out of the data set. So that gets us a refined list of AEC specific terms. Any questions about this or should I keep going to kind of the next step?

So you have two models, you have generic and then AEC specific. Do you eventually see it also being like a third firm, level one for the ACME strategy version, I want to be able to find that stuff too. There's going to be repetition in things like that in all kinds of meetings that get recorded. And it seems to me like if I'm in a firm that could be really useful, that's not profession wide or industry wide, but just firmwide.

Yeah, it's an open question. I mean, I think, I mean, it may not end up being a third model, but to kind of get at that, how do we blend the kind of the AEC specific terminology with the firm specific terminology? There are a couple different techniques to do that. We're exploring a couple different ones. But in general, what we're seeing is, we're seeing that this is going to go pretty far. The simplest way to say it, okay.

If that is specific to the firm and it's in their data, it'll pick up in the vector search anyway.

In the search it'll pick up. Yeah, sorry, the transcription one's the trickier one. The transcription one is the one where I was trying to avoid doing this, but I'll just do the short version of it. There is an option to maybe send phrases which are specific to that firm along with the video to get it transcribed. There's questions about building a firm specific model, as if there's enough volume to even make it worthwhile. But it's a good question. And obviously with anything because of context.

And the vector that you were talking about, right. It's like, yeah, yeah, I could see it being really useful.

So this is the thing that we missed, but we started figuring out really quickly that we missed it was the common terms that have AEC specific meaning. So, you know, when we applied the filter on this slide, we knocked out all those terms, which is fine, but we also knocked out terms like plan, section and elevation, which mean different things in common English. Right. Drawing, layer, model, core, flashing, pitch, program, specification, transmittal. Like these are words that exist on those common unigram, trigram and bigram lists, but that we really want to understand deeply what they mean in an AEC specific context. So we're using a variety of techniques, both automated and manual, to assemble this list of the common terms with AEC specific meaning. And then the last piece is the acronyms. So this industry is notorious for that. Right. So we've got project phases, we've got square footage and building codes, we've got all this kind of more sustainable climate type acronyms. We've got kind of more contractual or Business related acronyms like there, this is just representative, this is an all. So we're using a mix of automated manual processes to collect these. And this is helpful for both transcriptions, but then also in the search side, this will allow us to automatically expand search queries. So for example, if someone says just searches on cd, we'll search on cd. And Construction documents is a simple example.

This is great because I see this is where transcription services fail all the time. And even in the same conversation that it's transcribing, it'll transcribe something in multiple ways. Sometimes it'll actually write the number, sometimes it'll write the words that make up the number. For example.

Yep, I'm actually going to get to an example where. Yeah, it gets even worse than that. So now we're trying to acquire context. So this is a pretty simple step. So for each term that makes it into our terms of interest list, we're extracting and de identifying sentences from content. So for example, these are just gonna be Revit examples. You know, revit's used in coordination. It allows the team to do blah blah blah. Or revit can generate 3D models which helps us streamline design and create more visualization. Here's how we change and update changes in the model and so on. Like we're just really trying to understand how Revitz gets used in sentences. And the reason that's important is, you know, I, I did a video a couple months ago kind of talking about when we first started down this road. And this is what we found. In the early days you already gave example Evan of Rabbit and Revit. So you've seen that too. This, we saw the same thing with Deltec, like two words, the software company in Austin versus the AEC Deltec. So and there's a lot of these kind of things. So how did our transcription model get this right where the generic model didn't? It came from ingesting and being fine tuned with all of these sentences. So that when it sees a new sentence like this, we're going to be offering a new course on coordinating blank models in the, in the summer it knows to drop in Revit and not rabbit. Right. Because it understands in this context now if it's about I don't know who's eating the lettuce in your garden, like maybe rabbit's the right choice for you. But like clearly in this context Revit is the superior answer. So that's kind of. We determine the terms, we acquire context. The fine tuning piece is super simple. We take all those sentences we've collected and we send them into the model and then through a process called fine tuning and that gets us our AEC transcription model. That's step three. So this is where we are now in the process. And so what we're creating are what we're calling benchmark videos. So these are special purpose videos that we are making to contain terms we're calling benchmark terms. And those are terms that we expect a generic model will fail at. And we'll use those as criteria for evaluating the success of our model. And these aren't all the benchmark terms by any stretch of the imagination, but they are representative of kind of two typologies. These acronyms on the left are acronyms which you actually pronounce as words. So ashrae, bim, SQL lead, nepa, noma, which are different than initialisms like, I don't know, FBI or CD or sd. Generally the generic model does okay with SD and CD when you're enunciating the different letters, but it does really poorly with something like bin. It has no idea, it thinks it's Ben or it thinks it's some other thing. So the acronyms are important. And then on the right, when it's these AEC specific terms it's never seen before, then it really kind of struggles. And so those are what we're really trying to isolate is these benchmark terms to make sure that as we, you know, upload those into the model and we generate the AI generated AEC transcripts, we can compare them with human created transcripts of those benchmark videos to. And we know the human transcripts are 100% accurate, right? So then we kind of just try and find the delta and figure out where we're missing, which lets us then go and under improve the AEC transcription model. So, you know, in alpha we'll probably get to at least two or three generations of this model to get to a good place. But that means fine tuning with more context or in some cases fine tuning with custom speech clips. So it may be for Ashrae, like it just needs to hear ASHRAE said out loud in five different sentences with a known good transcription along with it. So it starts to understand that phonetic combination in that text, if that makes sense. So I'm assuming we're going to have to do some additional fine tuning with custom speech, which we then deploy a new version of the AEC transcription model. And then the circle is complete and then we go through the upload cycle and the comparison and we get to a Place that we feel pretty good. And that will kind of take us into beta just quickly. Beta. We're going to get qualitative feedback from beta testers. We're going to continue doing qualitative reviews. We're capturing every transcript edit that people are making. So our video transcripts, you can edit them and improve them. And so we'll be able to look for patterns and say like, look, we keep missing this word over and over and over again. Let's improve the model. And again, doing that through more context or that custom speech.

It seems like there's an opportunity here to present like during that QA process to actually see if the transcription. It seems like it could ask a reviewer what's the right option here. It could actually identify those. So that whole fine tuning thing is actually just. It's overseen by somebody who knows the answer to help it, train it.

So, Evan, are you talking about the transcription piece or are you talking about the search?

Talking about the transcription piece. Right. Because if somebody says ashrae and it's like ashtray or whatever, it's like, did you mean, did you mean this or did you mean this? And, and all I have to do as a qa, you know, fine tuner is just click a box instead of going through and reading it. You know, I'm just trying to think of, of ways to streamline this process for. To streamline the process because it also seems like it could be good at picking those kinds of things out.

I thought you were talking about this step where we're reviewing the transcript edits. I'm like, in this step they're telling us what they want it to be. But you were talking a couple steps earlier. That makes sense.

Along those lines, we're hoping this summer to build a. I'd really like to, you know, I keep using the word gamify internally, but it's like what you really want is a quick way that, you know, if, if we are making suggestions for things to give somebody, say, here's what we thought this was, but if this is something that's better, would you pick another word for it? Pop those up and just make it like, okay, I didn't have to do a whole lot of thinking and then put that back in the, in the feedback loop where, where those interfaces are and when people are willing to do that is an open question to figure that out.

But it is totally. It is. Yeah. We're definitely going to have to work on our search interface. Right. People need to give feedback. But like, is a thumbs down enough.

To help us improve or of a lot of this early training. You know, I think to the conversation earlier, Chris, this is where, you know, I would, I'd like to think that some of the more senior people in the industry who have the best knowledge would see this as a way of like, hey, I can really help the next gen of this because they can pass on a lot of knowledge and information very quickly. Right, right. To knock some of this out. Right?

Yes. Yeah. And I think that, you know, it's interesting, like I got into this conversation with our research council yesterday. There's a, there's a element of it which is kind of. Well, on the very short term there is like you leverage yourself more right. Through making synthesis, be able to answer questions you would like in a very naked self interest way. Like you make it so that people aren't coming to your desk and ask you the same question over and over again. And that's great. I think in a legacy perspective though, it's kind of like this, the Thornton Tomassetti story is like that. It's like, oh, like the impact I make now could still help the company run well after I'm gone. Like, that's decades. Yeah, that's amazing.

Yeah. Maybe there's a acknowledgement piece that comes with it. Right. That, you know, this stuff was trained by these people.

You know, it's like, yeah, interesting. I like that. That's cool. Okay, beta. We then deploy. It looks pretty much the same. So I'll keep going into continuous improvement, which is really my last kind of section. So the. So we imagine like during alpha we're going to probably build two or three versions. We imagine during beta, based on feedback, we'll probably build two or three versions of the model at least. Once we deploy, we imagine we'll build two or three versions. Like we'll be rapidly going, but at some point it will kind of settle down, we think. And so then we get more into like a. Every six months, plus or minus, we're probably building a new version of the model because we get improved models from Microsoft. So, you know, they're going to improve the underlying speech model. They're going to add new functionality that lets us do different things. You know, English continues to evolve and change as a language and so it will bring in new, new lexicon. So we'll want to do that. We're going to add more community AI members. So we launched the community AI program about six weeks ago and we have 56 clients as of today already signed up, which we are really happy about. So, you know, in context, ka. You know, we're getting near 140 clients. So we're, we're, we're approaching 50% of our clients, which was our goal for the end of this year. So I kind of have every confidence we're going to zoom right past that. So as we add more community AI members, they bring more content. We're going to poke in additional data sources like videos and profiles and potentially some of those search connectors to bring more content to train the model. People are going to be adding new content and that means new terminology, new context, all that kind of stuff. And then, you know, we'll continue doing fine tuning. So that's, that's really kind of the, the view from late April. In 2024, you know, in October, I might have a lot different view on how this is all playing out, but this is, this is the best I know right now of, of how we're doing things.

Chris, just from a business standpoint, are you going to differentiate and what people pay for the product about those two models or. It's basically if you want to opt in and participate, you get better information by opting in. It's a different way of paying. True.

It's definitely the latter. Yeah. So there is not a price increase for people that want to use our AI functionality. I say that with like 95% confidence. And it wouldn't, the delta wouldn't be because you were in the AI program or not. You know, we think our inference costs are going to be fine. We think, you know, like, in terms of like the infrastructure costs, like, we think we're okay. It could turn out that we're, that it. We have to kind of COVID our cost change there. But I, I don't anticipate that's going to be a major.

The healthy way to think about it that, you know, you're going to get if you contribute, it is the community, as you named it.

Right.

It's like the community aspect. If you contribute, then you're going to get better info out of contributing. And if you don't want to contribute, you don't get to participate in it. Right?

Yeah. And people can always opt in down the line, you know, and I think one of the, you know, it's been a campaign. Like, I have offered to meet with any one of our clients. We wrote FAQs, and you can imagine all the documentation we produced. I have met, I've offered to meet with any client that has questions and wants to talk with them multiple times if they need to Go through it. I really want people to understand what they're signing and to feel good about it. And if that means they wait six months or two years or they do these high end embassy buildings and there's no way they feel like okay, signing the contract because they're worried about their contracts with their clients. No judgment from us. Like we have an option that you can use that you can.

I think it's, you know, there's other examples where this, this industry has tried to get cross firm collaboration, but usually it was, they actually had to make some effort and do things and then, and then it gets lopsided because one person feels, you know, one firm feels like they've done more than the other. But I think with what you're talking about, it's really, hey, this is just a byproduct of making your info and data available to help train these models on. So I can see that you shouldn't run into those same kind of challenges with, oh, I put more in than, you know, somebody, somebody, somebody, you know got more out of it than they put in. It's like, well, that's kind of hard, really hard to measure in these kind of.

Sure, sure is one thing that I keep thinking of is, and you guys aren't guilty of this, you, you both have products that have to be deployed firm wide from whoever makes that decision leadership, it goes out the users of the firm. There's a lot of other SaaS and AI companies out there that are just goers, right? And they get into companies that way or people use them at their companies and maybe they should, or maybe they shouldn't. Like this is how Dropbox became a thing, right? It was like everybody I had to share files, right? And it was, if I have any kind of admin rights on my computer, I can use it. Well, these new SaaS programs, these platforms don't require. All I need is a web credit card, right?

And so in some cases it's not even a credit card. In some cases it's free up to 10 users and totally get like a beachhead going. Yeah.

Or it's, or it's really in quotes free, Right. Because it's just using whatever you use. There is a transaction of data happening here. You're providing it with something that it can then use to provide you the answer, but also train the model. Right. So my question is like when the firms don't opt into your thing, Chris, and they don't move fast enough for the users who are responsible for getting their job done on a Day to day basis. And they want to take this into their own hands, right? Like that is I think part of the conversation for these firms as well. It's not like you have to do this, but it's like, but, but understand this is the trend in the industry that we're seeing. These companies are going straight to the users and saying, look, you can just go to this address, type in your question, get the answer. And this is something we've talked about on this podcast, which is governance and ethics and all these things with AI. And the companies may be even really slow on the uptake of providing that information of why they've made the decision that they've made to their staff and what you can and cannot use explicitly, no matter what. Here's why I think that this is all part of that conversation that needs to be happening. Because like firms that are intentionally slow in deploying software that gives their users definite advantages on a day to day basis, like the users are just going to be like, okay, I'm just going to take this into my own hands. Like this happens all the time, right? I'm just curious from your standpoint, are you having those conversations with firms? Is that part of the conversation? I should ask.

I'll take a swing at it. I'm interested to hear Randall's take. I think a lot about the diffusion of innovation curve and recognizing that you're going to have your early adopters, they're going to move very quickly on this stuff. When we announced the AI, I swear we have people sign up so fast in some cases that I'm not totally sure they could have read everything in order to sign up that fast. And I'm like, are we sure?

Right?

Are you sure you know it but like I get it like and you know they. And it's also trust in ka, right? Because we have long term relationships. You start getting into early majority, late majority laggard territory, those folks are going to do their home. Like the early majority and late majority are going to do their homework and they're going to want to see value. Like people signed up and I'm grateful for our community people signed up on this pre software like they haven't been able to like see the generic version. Understand what would be better if we have aec. Like they can connect the dots and they believe it for people later in the diffusion of innovation curve. They need to see it and in some cases they need to see other firms going first to feel comfortable. They need to, they need social proof and whatever you want to Call it. So I think that's one angle to it. The other angle is like some of our clients have pumped the brakes a little bit and are pulling back anything AI related and kind of forming, you know, a governance committee or some kind of policy or something like that. And so what we've heard back is like, we like what you're doing, but in order to be equitable across the company, because we've said no to other AI projects, we have to have this process in place first to evaluate this and understand what we're evaluating. So I think that's really fair. I respect that. You know, going slow and being fair. And I expect under. I also, like, if you don't understand, some of them have been like, we like what you're doing, but we don't have time right now to understand if we can sign up. And so we can't sign up. And so, like, that's. That's cool. Like, I understand firms have a lot going on, so it's probably just a mix of all that stuff. And I think what we're going to do is keep building and releasing and showing value and iterating and communicating how things work. Like that presentation I just shared at the end around how we're actually building a transcription model. The reason I built that presentation is in those meetings I was taking with clients, I could feel this anxiety about what they thought they were sharing. And then I showed them that it's like Acoustics Abutment, Autodesk, but it's kind of thing. They're like, oh, my God, it's totally innocuous. We can totally sign up. So, like, they need to see that. And so there's another wave that just.

If you look at my end, we do these infographics every year. Chris. Which are like the most searched terms right across the platform.

Right.

My joke is always. It's like if you ask a third grader how to build a house, it would be chair. It would be those words light, like, hey, this is not proprietary information.

This is like, yeah, yeah. So some of it's just building the trust and showing your. Showing your work, you know, which we're doing and that. So I don't know. What do you think, Randall?

You know, forgetting about the AI component of it, you know, what we've seen on, you know, with firms deploying our technology in the firm, a lot of times it's deployed. You know, I always describe it as it's deployed from a very command and control. Right. The goal of the firm is we're. We're as a firm going to do better and we're going to do, you know, we're going to do this, we're going to get organized.

We're going to get organized.

Right. And you know, there's always this kind of top down, you know, kind of start of the project. But then, you know, the healthiest customers that we have are the ones that also figure out that really the best information sometimes isn't from the top down. Both you've got to have a combination of grassroots bottom up and top down. And you've got to have technology platforms that support both of those kind of coming together. And so, you know, we're the work that we're doing along the same lines, you know, of the kind of technology that you're implementing. Chris, we're doing similar experiments and things inside of avail and one of the things I think that is, you know, I'm trying to wrap my head around like where do we fit? And you know, where should we fit? Because everybody's working on these same kinds of things. You're going to get this coming from 100 different angles. But one of the things I'm most excited about is that in the end you're not going to have a, I don't think you're going to have a chatbot. I think you're going to have a thousand chat bots that are tuned to very specific things. Right?

Yeah.

This is why the agent thing is so important, I think. Right. Is because I can't even keep the Rolodex of all the AI chatbots that I need to talk with on my just for today.

But the models that are feeding those behind the scenes, the info that's feeding those models, to me, that's what's interesting about where we sit is that the people that have that information, we've got a content management platform that let them organize and manage that information. Right. And curated and you know, keep the model fed with the best information. And you know, we, we fight, everybody fights the same stuff. I mean it's like we use Chris, we use HubSpot for our, you know, our CMS and, and ultimately we use their knowledge base and it drives our, you know, the knowledge base that our customers use to find, you know, help documents on it. So we've been feeding our own chatbot on all that info and I can go in there and ask it questions and it's, you know, pretty good. And then I'll ask another question. I'll be like, you know, that's not the best way to, I Know that's not the best way to answer that question. So then you gotta. Now you gotta have an initiative that says we gotta go back and start plugging those holes with better information. I've become interested in this, thinking about, you know, you want to build in some kind of decay. Like some information should decay over time, Its value should decay or assume it's going to decay over time or expire. What drives that does time drive that does some other, you know, shift or change. And how do you, how do you now add that information so that over time those things start to drop off at the right time? You know, there's.

Versus which stuff is evergreen.

Interesting.

Just because it's old doesn't mean it's not dangerous.

You know, this is why, Chris, probably you and I are the kind of people in heaven too, that like, like these kinds of things. Because it's a technology problem, it's an intellectually stimulating problem to try to figure out, say how, how would you do this to make it evergreen and actually work over time? So.

Right. And from a, from a human perspective, like one, one approach, like, if, if it's true that like, some things are evergreen, some things expire in six weeks, some things are good for two years, some things you don't know, some things you know are good. Like the person in the, in the situation of creating that thing. Like, what percentage of people are going to have that kind of mindset and that kind of like thinking in the future that, like, when I make this thing, I know what, man, what did Louis Kahn said? Imagine you're building in ruins, right? So how many people creating content are imagining their content in ruins? Like, I don't think that's a lot.

I had a friend, an engineer who was an engineer who worked for a company, a corporate company, was an engineer inside their team. And he told me the story, this goes back years ago, but he said, you know, they started having. They started. They had to invoke a rule that said anything that was like, talked about in the meeting, if it was over 90 days ago, is irrelevant because they would have people say, but nine months, a year ago in this meeting, you said this. It's like it doesn't matter. The context has completely changed since then. It doesn't matter. It's garbage.

So I wonder if that gets into. We talked a little bit about filters and kind of focus. And I do wonder if, like, that is one of the aspects of, of how those filtering, you know, biases. And I mean, we do this already in our search index. We Wait very heavily. I shouldn't say very heavily. We wait, we take freshness into account when we return search results. It's not the only factor. But some of this can be done algorithmically. But I think some of it is going to be somebody who cares, you know, reviewing content.

I think part of it too is like if you're taking in data from all these different, you know, written and speech and video and it's like corroboration matters too. Like if you're hearing the same sentiment in multiple. Yeah. Signal from different sources, okay, there's still a freshness to that. Whereas, okay, somebody said it differently. Something might be changing or it might just be that person doesn't understand it like these other people do. And so, yeah, all that, it really complicates the situation potentially. But it also, I think there is a way to actually do that. Right. Where you can, because of this idea of freshness and kind of signal across mediums of communication and capture. It does sound like it's actually doable.

I think this is a golden era for knowledge management. I mean, knowledge management's been around, I mean, probably since like Hammurabi and like writing stuff down on tablets, but like since the mid-90s is when it really took off and there have been a couple waves. I think this is the third wave of knowledge management. I think it's going to be a golden era. Like, I think, like, because of all the things we talked about today and that value being so much higher in this kind of gravity of like pulling, like you said, Randall, like, you want to, you want to plug those holes because, you know, if you do, this is what you're going to unlock. Like, I just think that's where we're headed. I think it's going to be a good run for anyone that likes doing this kind of work. It's going to be interesting, great stuff.

Well, maybe that's a good place. I know we're. This, this was a good two hours spit. It's a good, deep conversation. We could probably spend two more. I love, love talking about it and thanks for sharing. You know, everything that you guys are working on, it's exciting where that's all going. And, and you know, I, I was, I teach this class. I had my last class in the semester in person yesterday. And you know, I was just telling the class they're not part. They're engineering and business students but at the university and, but I was telling them, I'm like, you know, a lot of the stuff that I work on, I get Joy out of because I truly hope that what I'm working on can make an impact on the industry. And I know you feel the same way. You know, both of you guys are feel the same way. And it's like, look, this is why we get up every day and hopefully stay excited about it is there's great opportunity to move the industry forward. You know, the comment made earlier about that this industry is maybe swimming in an order of magnitude more info and data than other industries just means that these, this is going to be vitally important technology to get implemented here.

Unstructured data.

Unstructured data. I just want to. I appreciate you guys creating this forum to be able to go in at this level of depth and honestly this kind of like level of technical like discussion. It's not that I mean I can't go to most conferences and talk about this stuff. So like, you know, so this is, this is a nice to have a space and you guys are so good to talk about it.

We can geek out. Maybe we need to start doing these as like the cocktail hour and we can just hang out forever and drink a beer while we're doing it. Right? Yeah.

This is the podcast for nerds. And I appreciate you, Chris, for preparing so much for this because this truly does get to the mission of this podcast, which is, you know, I characterize it as the director's commentary track. And look at us, we're movie length episode here. We've got the director's commentary track. You did the deep dive. It was conversational and really rich with information. So I feel like it's kind of checking all the boxes on what we wanted to do with this podcast and it's been a fantastic episode and conversation.

I really enjoyed it.

I have a feeling to have you back on before too long too to see some of the fruits of all this.

Right. We're not going to stop. We're on a tear right now. So I can't wait to show what are we doing in the future. Appreciate you guys again. Thank you so much.