Behind the Scenes: How We Built Synthesis AI Search

Okay, so as I said before, if you just joined. I'm Christopher Parsons, I'm the founder and CEO of Knowledge Architecture and I am joined by Sean Mann, our director of Software engineering. Hey Sean. Okay, so if you don't know about Knowledge Architecture, we're a software company based in San Francisco. We help firms build integrated knowledge learning platforms. The name of our technology is synthesis and we work with over 150 firms currently and growing synthesis. And when I say integrated knowledge and learning management platform, it's got three main legs of the stool. The first is the intranet. This is how we started the company, by building intranets that were hubs for firm wide knowledge and information. Last year right about this time we introduced Synthesis AI search. So bringing AI into the workplace and we're going to talk about this a lot today, so I won't spend much time on this here. And then the third leg of the stool is Synthesis LMS which went into private beta at the end of June. And this is another kind of key part of our strategy to help build a one stop shop for firm wide knowledge and learning. People know where to go to get answers to questions, get learning experiences, connect with their colleagues all in one place switches synthesis. We are integrated deeply with technologies that people care about like Deltec, uninet, open asset, AEC360 and more. This will actually be a big part of the presentation today as we start talking about how Synthesis AI search interfaces with data that we draw from these different platforms. So what we've been working on for the last 16 years, we started in 2009, is bringing knowledge management to the AEC industry. And the way I describe it is it's the practice of using people, process technology and culture to help your firm make the best use of its knowledge. So we connect people to the right knowledge, we connect people to the right people and we do so in the flow of work. Right? That's when it's all humming and it's all really working well. That's what that looks like. So I think we're in the golden era of knowledge management in AEC. So I've been working in this ka 16 years old. I first got into AEC and knowledge management about 25 years ago and, and this is the most fun I've ever had working in knowledge management. I feel like we're really, AI has really unlocked this new way that we can, you know, we've always been good at sharing knowledge and capturing knowledge, but retrieving knowledge and retrieving it in the flow of work when somebody needs it, exactly what they need, that's always been the elusive part and the hard part to get right. And I feel like what we're starting to see over the last year with AI search and synthesis and AI more writ large in knowledge management and is that retrieval piece and access is just totally transformed. And we're very excited about that. So we've got three parts to our or four parts to our presentation. I'm going to start part one talking about how synthesis AI search evolved as first a general knowledge base search. It then evolved into being an advanced employee search. It then evolved now into becoming advanced project search as well. And then I'm going to take a look in part four at what's next. So Sean and I gave this presentation a few weeks ago, three weeks ago, at a conference. And our title for this presentation then was as Little AI as Possible. You might be thinking, well, what do you mean? This is where it's a presentation about synthesis AI search. Why as little AI as possible? Over the last couple years, you might have been hearing these terms float around. Probabilistic versus deterministic, right? Traditionally, software that we built was deterministic, meaning we write code. When you press a button X happens, the next time you press the button X happens again, you press the third time X happens again. It is if this, then that at its most simple kind of reductive form. And that is how we've been building software for a long, long time. AI has come in and it's different. You don't always get the same answer when you ask an AI a question. Right? It's probabilistic. You can do things like grounded in knowledge to kind of have it draw from a certain set of facts, but it's going to potentially answer a question differently every single time. The facts shouldn't move around, but the answer will move around. And that nature, that kind of quality of AI is what's both maddening, but it's also what is so transformative in terms of how it's changed the way that we can build technology. And so what our view is at Knowledge Architecture is we actually want to bring those two worlds together. We want to take the best of AI in terms of its ability to be creative, have intelligence, understand nuance and context, and pair it with deterministic technology that helps kind of constrain it into guardrails and focus on having AI do what it do best, but having kind of more traditional software approaches do what they do best. So that's why when I say we try and do as little AI as possible. We really believe in the ability of the AI to create great solutions, but we want to be very careful on when we're using it. So with that context, I'm going to take you into part one. General knowledge based search. So the design goal for version one of Synthesis AI Search was to improve user experience for just general knowledge based searches. As an example, holidays. So if somebody searches on holidays in Synthesis AI search, what we do is not just return answers. If you haven't seen Synthesis AI Search, we return a search summary. Right? It's an answer drawn upon and grounded in the knowledge that's in your synthesis intranet platform. Right. So it could be from pages, it could be from posts, it could be from documents, it could be from videos. So somebody can get an answer, but we provide citations so that people can go check the underlying information. Another example of a general knowledge based search might be something like what's our 401k match? Great. Instead of having to just get a link back to the 401k page, we actually answer the question. So we match dollar for dollar up to 3% of your salary. Or it might be something like a senior PM bill rate. Right. You could go read the page or the document that has all the bill rates and go find the senior one. But why not? We should just return you the answer that you're looking for and if you need to drill in, you can kind of follow the link and go do that. Or we love this one. Why is my revit model slow? So the answer to this question might actually live across multiple different resources on your intranet. It might be in a page or a video or a post, it might be in a document. And so we can stitch together these answers drawing from multiple sources. There's one more recommended window to wall ratio. So our answer is that the recommended window to wall ratio is between 20 and 40% based on the content on the Internet. You can see in the bottom here that I'm calling out different citations. So if I click on the Windows and Doors page, you can see that that's where we got the 20 to 40%. And if I click on standard calculations, you can find out how one calculates a window to wall ratio. So this is what I mean by taking information that lives in multiple sources, pulling it together and providing the citations to the underlying information. We also one of the goals was, as part of using AI is to not just give somebody everything that matches this query, but really bring them the top most relevant results. So there's A piece here on the sidebar for top results. And then there's an opportunity at the bottom to ask follow up questions and engage and have a conversation with your knowledge. So this is all in version one. The vision of what we wanted to build and have built. The last piece I think can kind of tie together what we were trying to do in version 1.0 is if I search on something like twinmotion and there's no content on synthesis, we tell you that there's no information and your users have the ability to rate those queries. So if they click rate, they can rate it one star and they can say, please add information about twinmotion to the intranet. They submit that rating. I'll come back to what happens when they submit that rating a little bit later. And what I want to share is one of the unique opportunities that we had in working on this project is that we've been working, you know, with AEC firms for 16 years. So we have a lot of search history to draw upon that helped inform the design of the product. So as an example, this was just a subset that one of our engineering team members pulled. For me, it was like six months of data from 2024. I don't remember exactly the parameters, but you can see like this is how people search. First of all, the number one thing they search on is Holiday, which is interesting, and then Holidays is number six. That's universal across the community. But you can see if you squint your eyes more broadly, what you see is people are doing keyword searches. So they're not, they weren't coming into AI search writing extensive prompts or very detailed questions. They were doing keyword search because that was the way that synthesis search worked in the past. That was the way Google search worked in the past. All the different technologies that people used were keyword based approaches. And so we knew that we had to design synthesis AI search to, to handle these keyword queries in a really smart way. So as we got going, and I told you this was building version 1, but we actually got started with a 0.5 and the 0.5 involved using this technique that I'll call keyword retrieval augmented generation. And so let me talk about what that means. So when a query comes into synthesis, for example, what's our jury duty policy? We do a synthesis keyword search. So we're trying to use that query to, to do a keyword search and return the most relevant resources. So those may be posts, those might be videos, those might be documents. We then Send that to AI, a large language model, to then generate that summary with the citations and those top results that we saw. So this approach is called retrieval Augmented generation, or rag. It's retrieval because we do a search first and then it augments the AI generation of the answer. Right. So that AI is grounded in your knowledge and only your knowledge. This is the approach called rag. Okay, so why did we want to start with this simple approach where we're sending the whole documents to AI? It allowed us to do rapid UI UX prototyping, it accelerated alpha testing for us, and then it allowed us. And so alpha testing, I mean our internal testing and, and then it let us get to private beta or user feedback as soon as possible. So Sean's going to talk a little bit about rapid prototyping.

Yeah. So this is the real behind the scenes look at product development at ka. This is a screenshot of a Slack conversation between Chris and I where he's sending questions for me to run through just a toy version of this RAG structure that I had running on my machine a little more than a year ago. And that's how we'd like to start things is whenever we're going to work with a new technology or a new process, we like to start with the kind of, what's the most basic version that we can build to see if we can run into any core problems early on. And you can see already here we've got some basic citations going, we've got a nicely formatted answer. And so we know that this general retrieve answer search results and then generate an answer based on them, that that structure is working. After three weeks, we were in this next screenshot here where we've got our search summary formatted nicely. We've got some top results over on the side there. We could spend that time just working on our UI rather than working on that core technology. What did we do during that time? There's the basic backend in ui. This is just communicating with the LLM building out the way of displaying this kind of stuff. We refined our summary prompt so a lot of that is, this is like stylistic things, how we want it to write its answer, that kind of a thing, how it should be answering questions, firm personalization. So some of this is just letting the LLM know what your firm is. So if you ask a question like what's the vacation policy at CJP Architects? We need to let the LLM know that your firm is CJP Architects, that it doesn't start trying to use that as one of the search terms, citations, that's something that's really important. We wanted to get that right, not only how to get the LLM to use them and place them correctly, but so that we could then format them in a nice way on the front end. Top results. We're getting our search results back in order of their search relevance. But then when we present them to you after the summary has been generated, we want to show which ones were actually used to generate that answer. So we've got to put things in a different order based on the LLM's answer. Follow ups. Follow ups are a little complicated when we're working with something like this because we need to decide how many of those sources to send back to the LLM for each additional question that you want to ask about something. So we got to work through a lot of those problems. And finally search ratings, that is a feature in and of itself that we needed to build this process to let people rate these summaries and tell us what was working and what wasn't about these searches. And at the end of it, we've got this ui, this high resolution prototype that was not only useful for us, but we felt like that was good enough, that we wanted to get that in front of a few clients to see what their experience with it was.

Yeah, and Sean, good use of the word few. So it was three. So we, in this time last year, October of 2024, we went into private beta with three firms. Nobody had it live and it was this 0.5 version that was good enough for people to validate a lot of our assumptions and help us refine the ui. So based on those learnings, we moved to really building out the 1.0 I mentioned before. The first one was called Keyword Retrieval Augmented Generation. Now we're moving to hybrid retrieval augment generation and then we're adding in content chunking. So why hybrid rag and content chunking? So before the reason we called it Keyword is we did a keyword search and we would send the entire document or the entire video or the entire post. Well, that only worked with short videos, short documents and short posts. So we needed to chunk them down into smaller pieces to be able to handle really long, like 50 page, 80 page, 100 page PDFs or long video transcripts. And we'll talk about chunking in a little bit. We wanted to be able to navigate from a user experience perspective to directly to page sections, comments and video sections from those citations. And we also wanted to enable semantic search. So semantic search, which Is oftentimes called vector search. Allows people to search without getting the exact precise word right. And basing on the semantic meaning of their query, we can go find the stuff that's most similar. So standards or guidelines or pto, vacation, you know, holidays, those are all kind of semantically related topics. And so this semantic search technology allows you to do that without having to add a ton of synonyms to your synthesis intranet. It just does it basically using some technology that we'll talk about. So in this example before, where I was looking at the recommended window to wall ratio and I clicked on the windows and door page, what actually happened in the UX starting in version one is that we didn't only take you to this page, we skipped you directly down to section seven to tell you about the window to wall ratio. And the reason we were able to do that is we had broken this page down into chunks. So each section here 7 window to wall ratio, 8 occupant load calculations, those are individual chunks and that's what we actually searched on and sent to the large language model. Or in this example, if I search on Dropbox Headquarters summary, one of the citations down at the bottom is a town hall recording, specifically the project update section of that town hall. If I click on that citation, it will skip me right to minute 1231, which is where that Dropbox project update starts again. That was chunking. That allowed us to have that precise targeting of direction when somebody clicks on a citation. On the semantic level, if I ask a question, how do I connect to the wireless network in Boston? I can answer that question or synthesis can answer that question. Even though the word wireless does not appear in the title or the body of that underlying WI FI access page, all that says is WI fi. But semantic technology lets us understand that WI FI and wireless are similar enough that that's very likely what the person was looking for. So, exciting upgrade. So what does that look like from an architectural perspective? So, so that query comes in. What's our jury duty policy? We now do a hybrid search. So we're doing both vector a semantic search, but we're also still searching on keywords. And this time instead of retrieving the entire document, we're just retrieving passages or chunks from videos, pages, documents and so on. We then send that to the large language model to AI to summarize, generate that summary, as well as create those citations and additional results. So it seems like it's pretty simple. Diagrammatically. That was actually a pretty heavy lift technologically. And from an engineering perspective. And so doing that kind of 0.5 let us validate that we were on the right track before making this next big leap, and also validate that it was necessary, which we quickly figured out that it was both from the chunking perspective and the semantic perspective. Like people were hitting a wall with long content or synonymous content. So behind the scenes, when somebody uploads or edits a resource in synthesis, our technology does break it down into those chunks. It then runs it through what's called an embeddings model. I'm not going to get into embeddings model on this call. This is if somebody wants to go down a YouTube rabbit hole, they can. But essentially what this does is each of those chunks gets embedded with what's called a vector, with vectors. So think about this like DNA sequencing. A chunk comes in and we basically give it a DNA sequence. It's like this is what this content is about from meaning perspective, which when a query comes in, we also do that kind of like DNA analysis, like, what's this query about? And we try and then match the DNA strings between the query and all of those chunks that we have across all the content and synthesis and find the most relevant things that are most similar matches. So it's done with a lot of advanced math. It's super interesting and more to the point, it works really, really well and we're excited about that. So Sean's going to tell us a little bit about context engineering and take us through some of the stuff behind the scenes of how we built version 1.0.

Yeah, thanks, Chris. So we've been hearing for a couple of years now about prompt engineering, which people usually talk about as that's figuring out what's the best way to ask an LLM to answer a question to do something for you, figuring out just the right set of instructions. One thing that people who are building products with LLMs now are thinking about now is context engineering. And so you can see here's a definition from Andrej Karpathy, who's one of the OpenAI founders, kind of a thought leader in the field now, talking about how he thinks about what context engineering is. I think that the important part about this is that towards the end there just the right information for the next step. We can think about it in terms of these factors of instructions, data format, order and length. So instructions, that's. That is just like prompt engineering. How are we asking the LLM to approach this problem? Data, that's our sources format is how are we sending those Sources to the LLM order speaks for itself. What order are we sending all these things to the lm? And length. And that last one is important because every few months a new model comes out and you may hear about how this one's got a 1.5 million token context windows, something like that. This usually means that you can send an LLM the entire text of Moby Dick twice and it will be able to kind of think about all of that all at once. But something that we and other people working in the field have found is that the LLM does start to kind of lose track of sort of some of the instructions if you send it that much information. There's a great study from Chroma AI, who's a technology vendor. They sell their. They sell a vector database and they have some data behind this idea that the more that you fill up that context window, the worse LLMs get at actually following those instructions. And so we worked during this step to both get more relevant results for the LLM to work with. That's something that the vector search really helped with. But while we're doing this, we were starting with our keyword search, just sending the LLM a kind of big blob of text for each page or document that we were were returning because that's what we had for already doing our searching against for this step here, we decided that we needed to specifically format this information in a way that would be as useful as possible for the LLM. So we're going to look at some of the ways that we did that. A lot of this is making sure that we're sending the LLM all of the little tacit clues that a user would be looking at when they're looking at a page or some other resource on your intranet. So one example of that is page hierarchy. So we call these breadcrumbs. This is where on your intranet this page is. This is something that we weren't including in our search index originally because when this is part of the search content, that means that anytime you change something anywhere in this path, that we have to re index the whole page. So if somebody renames their projects and practice community here in this example, then we would have to re index this page. We ended up deciding that that was worth it. But because these breadcrumbs, they're effectively like tags for a page. So people will also often have information that's pretty important for interpreting the content of a page in the hierarchy that that page is in on your intranet. Headers, headers we were sending the headers as just text, but recognizing them as headers and what the kind of semantic meaning of a header is was important for this. Headers are also something that we're using to break page content up into chunks, contact blocks. So these are interesting because we weren't sending this content to the LLM before either. And this is something that we realized. There's a lot of cases where the best person to talk to about a subject isn't necessarily the person who's updating the pages. You might have a knowledge manager or somebody else who's helping to get this information into your intranet, but that might not be who you want your employees to go talk to about a subject. So we know a lot of people use these contact blocks in order to point people to a subject matter expert. And so we needed to get that into the content that we're sending to the LLM Publish date. That's another thing we want the LLM to start. We want the LLM to note the age of sources when it's going to be answering a question. But this one we ran into a problem which we'll talk about. We build out the first one here. So we actually, we just realized that we had to send the relative date. We started out sending the date September 28, 2024, in this example. And the thing that we ran into is, I don't know if any of you have had this experience of asking an LLM about something that involves dates. And it seems like the LLM thinks that it's 2022 or 2024, whatever its training data is for. And so we had to start reformatting things like page update dates so that they would be a relative date. So Rather than sending September 28, 2024 for this page, we would tell the LLM this page is more than a year old. And then we can give instructions to say, hey, please note, when a page is more than two years old, we can kind of tune that how. How we think we should. Headers. Some of this is about not only recognizing when we're breaking things into chunks, but making sure that the LLM can interpret things the way that people are putting them into the page. So we're formatting things like headers, and we do the next build. I think that's going to be lists, all of these. We're sending them to the LLM in a format called Markdown, which is a simple plain text format. That's something where the LLM can look at those and see, okay, what is A WET lab that is a header and that's got a meaning that the LLM can recognize. Same thing with list items here, that this is broken into a list that has meaning in and of itself. And so we want to make sure that we're passing that along to the LLM also. And then for the order and length, what we have circled here is an example of how we break a page up into chunks. So this chunk here, what is a WET lab? That's a good amount of text to send into a vector index. That's about, about the right amount of content to make a good chunk for retrieval purposes. But when we're asking the LLM to answer a question, we realize that we get better answers if we provide more of the surrounding context. So when we're indexing pages and other resources like this, we don't just index the content that's going to be used for that vector search. We also include the next and previous text blocks so that we can give some of that surrounding context to the LLM. Something else is that if you did a search for this page, you may find that the key components of a WET lab chunk here, the second block on this page, that may be the more relevant chunk that comes back from that vector search. And we found that if we send these text blocks to the LLM in the wrong order, if we send them not in the order that they appear on the page, that can cause the LLM to get confused. So even if we find that one block is more useful than another, we're going to always send them in the order that they appear on the page. And that's obviously especially important for something where there's a process or a procedure that a page is describing, where the order is critical. And then with all the surrounding context and reordering things, we need to remove the duplicates because that context window is so crucial that we don't want to waste any of that by sending duplicate content anywhere.

Great. And then Graham asked a quick question. The page update in synthesis for people that don't know, you can disable the last updated date in the ui. We still have that and still send that whether it's displaying in the UI or not.

Yeah.

Cool. That's part one, General knowledge based search. So then part two, we moved into advanced employee search. And this brings us into early 2025. Our goal with version 1.1 was connecting people to people. And so this I mentioned earlier, when you submit feedback, I said, I'm going to tell you where it goes later. So it goes Two places. First, it goes to the intranet team, the knowledge management team, whatever you want to call it. They see all that feedback that end users are submitting. So when somebody searches, that feedback gets shared. You can improve your content. It could be that there was knowledge that was missing. Something could be outdated, it could be confusingly written or, or duplicative, whatever it is. Like that feedback helps you improve your content, therefore you're improving your search. People are using it more, they're sharing more feedback. You're improving your content. You get into this really great virtuous cycle, or this great knowledge flywheel, as we like to call it. Also, when you submit that feedback, it comes to knowledge architecture. So this allows us to see patterns happening over queries. We start understanding where we need to make improvements in the software. And so that goes back. So as people are having better search experiences because the technology got better, now they're doing more searches, they're sharing their feedback. It is a very great, symbiotic, dual flywheel, virtuous cycle of improvement that we've been on with our clients for the last year using the product. So out of that feedback, we start learning that people are trying to do queries like this. Who has experience with wetland delineation? And based on the available information on the Internet, it said there's no employees identified with experience in this area. And I rate it and say, like, that's not true. I know we have several employees who've worked on these projects and they're in the employee directory. Why aren't you finding it? Synthesis. Good question. So we in fact got lots of feedback like that. This is just464 and this is an early screenshot I took when we decided to start working on this project. And so you can see people are looking for all the employees in their Chicago office or who's a notary, or who's on the board, or who has experience with wetland delineations. Like, we saw so much of this feedback and it was so clearly the number one thing people wanted to do. Once version 1.0 came out for the knowledge base search, the next big thing they wanted to do was search for employees. So we realized for the Jaws fans out there that we were going to need a bigger boat. And that boat. Well, let me tell you what was missing for the boat, why our boat was too small. To answer those employee questions right, we had to be able to give an answer that included the best things as well as all the things. So the best things is, if I'm looking for the WI FI password, or if I'm looking for a 401k match, like, I want the best content to get me the answer to that question. But if I'm looking for who has experience with wetland delineation, or who's in our Chicago office or. Or who's a licensed PE in Washington, I don't want three of them. I want all of them. And that was something that that current hybrid RAG approach I showed you before just wasn't designed to do at all. And so this is it, right? This is that hybrid workflow. You can see we're trying to retrieve the key passages, the best things, like it's literally architected to not solve the all the things problem. So we also wanted to make better use of the employee directory and employee blocks. We have good employee data, you know, based on integrations with deltec and UNET and Open Asset and so on, as well as these employee blocks on pages. So as an example of the employee directory, you can see that we've got department discipline office as well as a bunch of other data points. We should be able to take advantage of these in synthesis. AI search is what we were saying in the spring of this year. This is an employee profile. Again, you can see these fields about M rich information on a professional bio, a personal bio, skills licenses, certifications, all these things. In addition, in this example that Sean showed you before on WET Labs, we've got Ann Johnson here on the right as that employee block. We wanted to be able to factor that into a question on maybe like, who should I contact about WET Labs? Right. In the first version, we weren't able to answer that question very well. Okay, so what was missing from our boat is now. So we started with keyword rag. We then went to hybrid rag. Version 3.1.1 was agentic rag. Sean's going to talk about our first swing at this and then I'll talk about the revision, then we'll come back.

There's a truism that it's good for a computer programmer to be a little bit lazy. In true computer programmer fashion, I thought we would just see if the robot can do it for us. We set up this structure where we would send the question to the robot, to the LLM and let it retrieve. Generate a query to retrieve results, look at those results, and then if it thought they were sufficient, then write a response. With this example of who's our revit guru? We would send it that question and then it would first try searching for revit and get too many results or Try searching for Revit Guru. And if we don't mention the word guru, then it doesn't get enough results and then eventually get its way to something like Revit Expert, and then that would sometimes end up with it getting the right answer. And the problem that we saw with this was that it was really inconsistent when it worked. It felt amazing. It seemed like this whole AI thing is really going to work. We can just let the chatbot solve these problems for us. But that happened maybe 60% of the time, and that's not good enough for a search use case. So we really wanted to figure out a way to improve that consistency by constraining the scope that the LLM operated within so that we could provide more repeatability and make sure that we're generating good, relevant search summary results all of the time and not just 60% of the time.

Right. Which brings us to again, as little AI as possible. So we realized we have let this swing too far in the probabilistic direction or we have given too much responsibility to AI which it's not ready to handle. So we needed to claw some of that back and make sure that we're giving it good guardrails and sending it down the right path. Right. So we'll get into agentic rag. So agents has been the term of the year in 2025. Everyone's got a different definition. For me, it's pretty simple. AI comes with reasoning capability. It's got some intelligence, it's artificial, but it does have intelligence. So it can make decisions. What we do as both software providers, but then also you as users of the technology is you provide it with the grounding knowledge it needs to make good decisions or answer questions. We also need to provide it with context. Sean was talking about context engineering. So agents need reasoning knowledge, context, and they also need tools. So tools let them do things like access the employee directory, for example, in a way that lets them get all the things right. And so we need to continually build tools to help AI do its job. So this combination of reasoning knowledge, context as tools is to me what makes for an agent that can make good decisions. So how does that manifest in the synthesis? So now when a query comes in, this one's an employee related query who can help me with a drone survey. What we do first is we ask the AI to categorize the query. It's a very simple question. Is this more about a general knowledge base query or does this sound more like it's a question about an employee? Like it's a very simple question. And if it answers general, we then have it use that hybrid RAG approach we saw before, where it's trying to find the best stuff, right? To generate its answer. If it says this feels like an employee query, we then ask it to further categorize. Okay, so is it more like a search for an employee name? Is it more like they're searching on role or title? Is it more like they're looking for who has expertise with this thing? Or is it a straight up filter on the directory? So licensed PEs in Washington. And so based on its question, it will use different tools. It could then even go out to the intranet to supplement its knowledge, if need be, to then generate an answer with citations. So this is agentic because we're giving it tools, we're giving it context it needs to make decisions, access to knowledge, and it's also what we call a query tree. So we've built this tree structure, a deterministic tree structure, to help AI navigate and use its probabilistic capabilities more effectively. Right? But we limit the scope of what we're asking it to do. We're basically asking it to make judgments about things. And we give it pretty strict. We give it also assistive language to help it understand how to make those decisions. So now we can ask questions like who can help me plan a flexible WET lab? And we're back to that Ann Johnson example, like on that Wet Labs page where we saw her as the employee contact, Right? Or who are our PEs in Washington? We can answer that question. And in the way that we designed the ui, we're also providing explainability. So you can see the reason we said it was six is because in the directory we took licenses equals professional engineer and license by state equals Washington or who is our mass timber expert? Right. And we're able to in this case cite intranet content. So it could be a video is the reason why. Or who can help me with Dynamo scripting, Right? And then this might come from an employee profile. So this might be in her AMA resource for her skills, perhaps. So we've got all this diverse information, some of it in the structured directory, like the employee directory, and some of it unstructured in pages and posts. And so the goal for this version, the advanced employee search, was to take advantage of all of that knowledge and get connect people to the right people, revit container files. So one thing that we wanted to do here, so this is a knowledge base query, which is what we went after in the beginning of synthesis. But what we always wanted to do is to do both, if we could. So in this example, we're answering your question. We're describing what a revit container file is, but at the bottom, we're also suggesting that you can contact Brianne Walton, who leads this area. So in the best thinking back to my definition of knowledge management, where we want to connect people to knowledge and connect people to people, ideally we're doing both. And this is an example of, when possible, we do both for those knowledge base queries. Okay, Sean's going to talk about tools.

Yeah, so tools. So tools are how we're kind of putting the parts that our deterministic code is going to do into a box for the LLM to decide what to do, and then we handle actually doing it. Here's a simple definition of what a tool is. They always include text, description of what it does and what it should be used for, and then a schema. So the schema is kind of like a form for the LLM to fill out in order to execute this tool. So one of the ones that we were just looking at is our employee expertise tool, where the LLM just has to send a value, you know, a piece of text. Just saying, what field are you looking for somebody with expertise in? That's a simple tool. Next we're going to talk about one of our more complicated tools, which is the employee filter query tool. This is letting the LLM use the employee directory on your intranet. So the schema part of this describes what kind of filtering you can do in the employee directory. So that's verbs like equals contains more than less than that kind of thing. And the problem is that each firm's data is different. So different firms not only have different data in these columns, but you can map custom fields and that kind of thing. So people are going to have entirely different columns. So we can't just kind of write one description for this employee filter query tool and then use that across our whole client base. So what we did to solve that problem is we are generating per client field descriptions. So I kind of think about these as being a LLM version of the dropdown when you're building a filter in the employee directory. So you can see here, if you want to filter on office, you're going to get a list of your firm's offices that you can pick from. We want the LLM to be able to do that too, so that if you say how many people are in the SF office, it should be able to look at the available options and say, okay, SF probably means San Francisco, because I can see that here's the list of available offices for this firm. When we're generating these descriptions, we have to update them regularly and on demand. Because whenever you add a custom field or change a mapping, something like that, we want to make sure that we're keeping this information up to date for longer fields like professional bio, employee resume, things like that. This is some place where we use more AI. So we will actually take all of the values for that column, send that to an LLM and ask it to write a two sentence summary of what kind of things do people talk about in this field? What are some keywords that people mention there? And the point of this is that we want the LLM at query time to be able to read this description and see, okay, is this field a field that I should be using to answer this question? Is this going to have relevant information to this query? And then it's important to note that with these field descriptions, especially these summaries that's generated by an LLM and summarizing is something that they're pretty good, pretty good with. But we're not using those summaries to actually generate answers from your intranet. These summaries are used when we're building the query to go look for relevant sources. Sources. But those summaries aren't given to the LLM when it's time to actually write out its answer. So that way we're not kind of stacking AI on top of AI. There were a few other things that we had to do to really get the kind of results that we wanted to. And a lot of this is about industry specific terminology and that kind of thing. So licensing was a big one with architects. There's a lot of ways that people talk about architectural licensing. So people will talk in terms of licensed architects or being a registered architect at some firms. The field itself might be just whether somebody is AIA or not. So we kind of had to explain this broadly to the LLM to let it be able to pick the right filters. There's something pretty similar around Professional Engineer where people may not capitalize that in their query or people might just say pe. And we need the LLM to understand that even if it's not capitalized. We're talking about the, you know, the state license role of professional engineer here. And then some of this is client specific too. So you have internal terminology that might be specific to your firm. If you use synonyms, then the search synonyms feature of synthesis, then you can define something like what an Acronym means. So LDP is leadership development program. That way you can search for LDP and will bring back leadership development resources. Another one is some firms will refer to their different offices as studios. And so if we're going to handle a question like how many people are in the San Francisco office, if you call that field studio, then we need the LM to know that even though the query says office, we mean studio at this firm. And that's how we're doing that is by sending it these search synonyms that you can update yourself. Finally, a whoosh for a. Just do the build here, I guess. We are validating employees that are mentioned. So this checks any employee who's mentioned by name in the summary. So this is something that happens after the LL has written out the answer to your query and anytime somebody is mentioned. And so this could be something that's based on an employee's profile page, but it also could also just be from somebody being mentioned by name on a page or in a post, something like that. We gather those employee names, we look them up in your employee directory. Synthesis has a great list of name variations. So this is something like if you search for Kathy and somebody's canonical name in your directory is Catherine, then we're going to be able to handle that. And this lets us, as in this example here, flag when there's an answer that is referencing an employee who's no longer with the firm. And we try to use this kind of precise language about Darrow Watts could not be found in the employee directory and thus may not be a current employee. And an added bonus of this is that especially if you are getting an answer that's based on something like a page or a post where there may not be a linkified employee name in that, then we're going to be able to recognize where these employee names are in the summary generated by the LLM. And then we can add a citation so that you can quickly jump to that person's profile page and make sure that we show that employee in the top results at the. At the end of the answer.

Nice. Just close out employees. Going back to the screenshot I showed before of all the feedback that we got, this actually then turns out to be great for testing. So once we built the alpha, we could actually run through all this feedback that you personally on this call or that your employees have been submitting. And this became great test data for us to make sure that the product was operating at the way that we wanted to even before we got it into beta. So the Feedback forms that you guys fill out are instrumental in terms of improving the quality of your content, but also helping us drive the software forward. Speaking of that, once we kind of work through Advanced Employee Search, the next big obvious bucket of feedback was around Advanced Employee Project Search. So we wanted to connect people to projects, clients and consultants. That was the, the scope of what we wanted to do here based on the feedback that we were seeing. We also wanted to get better at connecting people to people. So what we didn't do in Advanced Employee Search that we have now added in Advanced Project Search is being able to use people's work experience. So I've worked on six of these NICU or natal intensive care units or whatever the thing is, then I can infer who has got expertise working in NICUs. So advanced project Search, I should say is in beta right now. It's an open beta. So if anyone's on this call, it's a client that would like to use it, just send us an email at support and we can get you added into the beta. And so we're working through beta and that should come out in a general availability once we're through testing. Okay, so back to the flywheel. What we heard was Project Search. There was a lot of stuff. Give me a list of employees who've worked on academic projects in the last 10 years. What's this Kresge project? Is this project an at risk type? On and on and on, like all these project questions. And we weren't doing a good job of taking advantage of the data in the project directory. So once again we learned that we were going to need a bigger boat. So this got us to the agentic retrieval augmented generation, which we talked about before, but we added in a disambiguation layer. So here's the project directory for people who haven't seen it. You know, there's filters on project type and client. If I click on one of those projects, I get a project profile. So construction costs, square footage descriptions. This is data pulled in from Deltec, open asset, Uninet, AEC360, etc. And in this previous design, this is the design for Advanced Employee Search. We just weren't taking advantage of that data. So we got back to the drawing board. Now we have a project query that came in. What library projects have we designed with atriums? We then categorize that query and instead of is this a general or an employee query, we've just extended that list that we're asking the large language model to categorize. So maybe It's a project, or is it a company? Or is it a contact? Is it a relationship? And one of the things that was really important that we added in was this disambiguation layer. So, for example, you can imagine a query like Ann Johnson. So is that an employee query? Maybe? Right. Is it a project query? Well, it could be the Ann Johnson Cancer Center. Is it a company? It could be Ann Johnson Structural Engineers. Is it a contact? It could be Ann Johnson, CEO of Ann Johnson Structural Engineers. Right. And so this disambiguation layer helped us make sure that we understood what the query was actually about and then bring the right knowledge to the AI to help make its make its decisions. So that was a really important piece that we added. But let's just say what library projects have we designed with Atrium's clear cut project query? Now, again, we're doing this query tree thing where we're giving it different tools and we're asking it to help figure out which tool it should use. Project number query. Is it a project number query? Is it experience query? Is it a directory filter? It then kind of analyzes that it will loop back. If it feels like it went down the wrong track, it can loop back to the beginning of the query tree and start over. It can also then search the Internet to augment its knowledge and then generate that answer with citations. So as an example, Canal Street Studios, right? This one was pretty good before we did advanced project search. But now things like project number queries, really simple, like that's the project number, we're able to find it. Or what's the project number for braintree in the form of a question, we just tell you it's 1804. Or please list our lead silver or higher projects. Great. So we've got 10 projects. If you look a little more closely in the explainability, we say that, hey, if it matches the filter, lead is silver or gold or platinum. Right? So it's understanding that silver or higher also means gold or platinum. And then it's able to show exactly where it pulled this data set from. What projects have won AIA awards. And there's one project, and there's a filter for project awards contains AIA or what projects have we built in Massachusetts? Again, we're explaining that matches the filter in the project directory state equals Massachusetts. Okay. Higher ed projects in Massachusetts, over 200,000 square feet. So this one is requiring multiple different queries. Right. And you can just see from the explainability how it's finding those two projects. Projects with automated humidification Controls. So this one's searching those project descriptions so that those rich descriptions that are in project, the rich information that's in project descriptions is really, really helpful. Because as much as capturing structured data in fields about your projects is really, really useful and you should be doing that, you can't capture all of it in structured fields. Some of it's just going to be a narrative description. So those project descriptions end up being really, really useful for advanced project Search. Okay, now we're looking at some of those relationship queries. So what higher ed projects have we done with Elemento? Right, and then I can see, well, we've done two projects that are higher ed and then that's connected to the company Elemento Mechanical Consultants, or what architects have worked on projects in Washington. So first we need to figure out who are the architects. We need to figure out which of those projects were in Washington and then we need to compare that set. So this is kind of searching across two different data sets or what higher ed projects have M. Davison and Ann Johnson worked on together. So we're looking for projects that are higher education where both of them worked on it. Again, this data is all in that project directory. It's just hard to do these things with traditional software filters and search. Like this ability to use AI to kind of synthesize that information and give you kind of unique glimpses into that data. That's a new thing that's unlocked with AI or let's just even take the degree of difficulty up higher. What higher ed projects have M. Davison and Ann Johnson worked on together with bcn? And now we're combining three things, right? The people, the projects, and then the company that they worked with also. So these are really powerful things for marketing, especially if you're doing, you know, some of the SF330 work or you're doing any kind of government work, like proving that you've worked with these people together in the past is really important. And that's why we went after this so hard in advanced project Search. And lastly we put math into it. So what's our average cost per square foot for higher education process projects? And you can see that we kind of not only tell you where we got the data, but we also, from an explainability perspective, show the formula that we use to calculate it.

Okay, let's build through this one quickly because the lookup tool, I think what's interesting about this is this was the first time that we gave the LM access to a tool that it can use at the beginning of this query tree process. Previously we would work through the query tree to get to one tool that we're going to use to pull our results that we're going to generate the answer based on. We realized once our tree got this large that we sometimes needed to look up information just to be able to go to the next step of that tree. But this lookup tool is kind of like search suggestions. This, you know, kind of autocomplete looking as you're typing, we're showing you some things that are matching, but again, making a kind of LLM friendly version of something that helps people in the ui. So this is going to help us categorize these searches that could be in any of these directories. Just like the employee validation. This is going to resolve nicknames and misspellings and that kind of thing. And then once we retrieve these, it is going to include this. You know, these, these are very short kind of stub results. Just saying there's John M. Is probably John Morrill Electrical Electrical engineering. We're going to send that to the LM for each of these steps through the query tree so that it doesn' lose that context that we got early on there. The Relationship Filter query tool is a really complicated one because Relationship Filter queries are always going to be different because we're going to be pulling in different entity types, we're going to be having different relationships between them. Sometimes we're referring to things by name, sometimes by filter. And so when we first tried to handle these queries, we tried just building one tool. So we would build a single Relationship filter query tool. And we had to use these words like result entity, include fields or related entity filters. And the problem was that even when we included a little helpful reminder to say to the LLM, okay, we think the result, like what somebody is looking for a list of here is going to be employee. It was still having a tough time filling out this really complex form with these kind of vague terms about result entities and related entities. We go to the last one here too, the field descriptions. If we're going to be doing a query like the one that involves projects and companies and employees, then that means we may be sending field descriptions for three different entity types. Those field descriptions can be pretty long. And we start to run into that issue that we talked about earlier, where the more that we fill up this context window, the less reliable the LLM is going to be. So what was our solution to all of this? We actually broke this tool up into two steps. So first we're just going to categorize what kind of a relationship filter query it is. So some of that is just, you know, what does somebody want a list of? What should that thing in the list be related to? Some of that is also how are we getting those related profiles or those, that initial list. So this is. If you're saying something like what projects have Ann and Bert worked on together, then we don't need to have the whole employee field descriptions sent to the LLM because we know we're just looking for Ann and Bert. We don't need to filter the employee list to execute this query. Then based on this, we're going to create a kind of bespoke tool that's specific to this query. So that doesn't include these result entity filter type vague terms. We're going to just call it project filters. And the LLM uses these terms much, much more reliably. And then we've got a little flowchart here of what this process looks like, going through the query analysis categorization first and then executing this query specific tool that is going to be easier for the LLM to kind of fill out that form. Calculations are, I think, one of the most powerful and fun parts of this. So as you may have heard, LLMs are still not very good at doing arithmetic. And so you hear these headlines about how LLMs are winning the Math Olympiad and that kind of thing. And that is true. But a lot of the models that are doing that take more time. They're this, you know, doing this reflection where they're chewing through a lot of tokens just to kind of get to the end of that process. We don't have time to do that on a search time scale. So what we're doing is we're giving the LLM access to some tools and things in order to do this math. So that's kind of broken into two phases. There's calculations performed during retrieving things from the directory. So that is something like getting the sum of the values in a single column. So like total cost is the sum of the field construction cost. And then we're also going to do things after that retrieval is done. So you could get total cost while we're retrieving projects and then maybe also get total area. And then if we want to get a kind of average cost per square foot, something like that, that's a calculation that we're going to perform after that retrieval is done. And so we let the LLM specify these two things. What information do we want that's just kind of an aggregation of data that's in the directory. And then what do we want to do with that data after we've retrieved it? That's these calculations performed after the queries and the calculations like all of our filters. Sorry, we can go to the next slide. That's another thing that we're going to explain. And so this is. You saw a lot of the examples where we try to show how we're performing these different filters. And that's something that we're doing with the calculations as well. And I think what's important to know about this is that the part with those explanations that's generated by deterministic software, that's coming from code, that's filling in a template with this is the filter that we ran. These are the calculations that we ran. And the reason we had to do that is because there have been a lot of efforts to get LLMs to kind of explain how they're arriving at different answers or what their thought process is. And the problem is that whenever you do that, that is another kind of text generation exercise. And so the LLM basically lies. Maybe that's too strong a word. But the LLM is not necessarily telling you the truth about how it is getting information. So we're taking on that role so that we can make sure that, you know, this is how we actually arrived at this answer.

That's great. So Susan put in the chat, we are going to run a little over to get to part four, which is what's next. So if you have to go, we're going to send out the recording and you can see it if you want to stick around. The what's next piece is pretty short and then we can do some Q and A. So we just wanted to share all this information with you. And I'm sorry that it's run a little bit long, but hopefully it's been worth it. Okay, so what's next? So currently we have 130 plus firms in the AI search beta. So just over a year and then 80 are live to all employees. So a few things are next. 1. Synthesis LMS search. So as most of you know, synthesis LMS is in public, sorry, is in private beta course catalogs and lessons and learning paths. All this exciting stuff about learning content, which is a separate conversation for another time. I'm bringing this in now to say, like all that firm wide, high quality content will be searchable using AI Search as well. So we're very excited to bring that content into AI Search too. Are we going to do more AI So I said as little AI as possible. But there are cases where we'll do more. Sean?

Well, first, here's what we're not going to do for now, and that is fine tuning. Fine tuning is something you might have heard about. This is basically sending an AI model a lot of examples of here's problems and then here's the solution that we expect out of that. That can be a really useful technique, but it's time consuming. And the problem is that that is really helpful for improving a process that's stable. And what we're seeing is that as we're adding capabilities, we're getting of questions from people using AI search. And so it's not time yet for us to be trying to make that kind of incremental improvement to that things that we are going to keep an eye on are evaluation. We keep adding this functionality and we want to make sure that with every new thing that we're adding, we're not taking a step back with some of the stuff that is already in a good place, a custom embeddings model. We're getting pretty relevant search results. But we could be doing even better, we think, if we had an embeddings that was industry specific so that kind of understood basically like synonyms, but in a AEC context and visual document parsing. So we do generate answers based on things like word documents and PDFs, but all that is just based on the text of those documents. Right now we can't take into account things like charts and graphs, let alone photographs or anything like that. And there's cases where that would be really useful. Composability, that's a kind of software engineer term. And that's something where we use that to just say how well can we take these things and combine them in different ways in order to make something else that's going to solve a different problem. So if we just look at this, the kind of leaves of the query tree here, where we've got these different ways of querying for projects, each of those is a tool that we think is pretty useful in its own right, and something that that could be used in different contexts if we are putting a different AI structure around that. So there might be other places where we're going to use AI to help us find the situations where we should be using those tools in order to help generate some useful information.

I've got an idea of another AI tool where we could potentially reuse some of those tools. Sean, let's talk about what's next in knowledge agents so knowledge agents is the next big thing for synthesis AI. This is a 2026 project, just to be clear to everybody here, and it is in design now. So I'm going to show you some early design work. We've been doing research for the last six months on this project and we're now kind of figuring out how we're going to design and build this thing. So I want to show you kind of the use cases and what we're going after. So back to what's an agent, right? It combines reasoning, knowledge, context and tools. And so here's an example. So you could build in synthesis a marketing research assistant. So what that would let you do is find firm approved positioning language and project intelligence, drawing from the boilerplate library and project directory. So what's kind of unique here, right, is you're specifying what you want it to do, kind of a job description, as well as you're grounding it in a subset of your firm's knowledge. It might be just the boilerplate library in the project directory in the design for knowledge agents in synthesis. You could also have these starter queries like you see here to both explain to somebody how they might use this agent, but also they can click on them and actually use them. If they're common questions or common queries or common tasks, you can just streamline that process for folks. So behind the scenes of this marketing research assistant, you've got a job description, so you're telling it what it's there to do. Your role is to do xyz. When responding, you can talk about the tone, you can talk about the format, like complete sentences and avoiding jargon, clarify technical terms, define acronyms. However you want this agent to behave, think about it like writing a job description for an employee. You're going to write a job description for a knowledge agent to tell it how it wants you to help people. And then the other part of that, right, there's more to it. But like two big concepts are the job description and then the grounding knowledge. So we're saying actually just answer only drawing from, in this example, the project directory and then a boilerplate text library that we've created. Or maybe you create a brand communications assist that helps you draft social posts and blogs and announcements using your brand voice. But it's drawing from content on the intranet to help you do that. LinkedIn posts or blog posts, whatever it might be. Or maybe you create a help desk communications assistant. So based on that grounding knowledge, that is all of the technology documentation on your intranet you can paste in a ticket that comes into the help desk. You can tell it in the job description, like, hey, here's the, here's how we talk to people, here's how we want to structure our answers. Be friendly and warm, Be technical, but not overly technical. However you want to describe it and then it can automatically generate the response for you based on that grounded knowledge on your intranet, in the job description you gave it, on how you wanted to answer questions. Or a video ops assistant. This is something we've been seeing people try to do with AI Search is put in a link to a video and say, generate a short description or suggest breaking this video into chapters or list the key takeaways or thinking ahead to the lms, like how would you break this down into a lesson or a series of lessons? So again, you can ground it in knowledge, give it a job description, and you can see down here on the bottom, you can also attach content to the agent. So maybe you just direct it directly to that video. As an example, you can imagine creating a revit assistant that's grounded in all your revit standards and best practices. Maybe it's grounding in the revit courses in your lms, your synthesis lms. And then this is an example of the starter prompts. Like these might be things that you actually just want people to come here and click, right? Because they're like common questions and you're trying to like put them in their, in their path. You could imagine creating a sustainability advisor, right, based on all your sustainability standards and your best practices and your codes. Maybe some learning content that helps people, you know, go to draw on the expertise of your sustainability folks without having them have to ping those people all the time. So it can kind of be an assistant to that sustainability team. And maybe the agent can handle level one and level two type questions, which lets your director of sustainability and the rest of their team focus on research and design and tackling higher level or harder problems. And also the sustainability advisor, the knowledge agent would be available 24 by 7, right? Doesn't need to take PTO, doesn't go on break, so it's available to help people when they need it. And lastly, one of the, and these examples have come from just talking through clients for this year and trying to like watch what people are doing with AI search that we think could be made even to the next level by constraining a subset of knowledge and letting you configure those job descriptions. Right? So these have emerged organically through feedback in the community this one we kind of like because it's an idea of seasonal agent. So most of your firms are in open enrollment right now. It's that time of year. What if you stood up an open enrollment 2025 assistant? Right. You grounded that in whatever it is, a PDF or a guide that your HR team had put together. And then people could come to this again, 24 by 7. They're oftentimes looking at benefits with their partner, you know, off hours or on the weekend. Right. So they can get answers to, again, the level one and the level two questions. Maybe in the job description, you actually instruct this to be verbatim, Right. You don't want it getting creative or kind of writing kind of these summaries. You actually want it citing directly from the open enrollment material. So again, you're able to tune and kind of like craft these agents to the specific use case that you're trying to solve. And I say seasonal agent because you could stand this up for open enrollment in 2025 and then either delete it or just kind of disable it for the rest of the year. And then when 2026 comes around, maybe you need to update the job description. You obviously want to update the grounding knowledge. But then people can use it again in open enrollment 2026. So this is something, as I mentioned, we're actively working on in design. This is a great time, if you guys have feedback or questions about this to send them in. And yeah, we really appreciate that. So that's kind of the end of our show here. I know we've got some people hanging around to ask questions, so I'm going to stop sharing my screen and we'll check out the Q and A panel. All right, Sean, I'm going to read this one out loud. This one might be for you. Can users encourage the AI to follow a particular tree by using keywords like employees or projects in the body of the prompt? Like, I know if I'm expecting a list of employees to return to my search result, is it helpful to begin my prompt with which employees are licensed PEs in Washington, as opposed to just what are our licensed PEs in Washington? Cool.

That's a good question. First, I have to say that we do try to teach the LLM. What are some common formats of questions that should be directed in certain paths along the tree? So something like, who are our licensed PEs in Washington? Is something that we should be handling correctly today, but it is definitely true that you can use some of those keywords to kind of direct what kind of an answer you want. And so an example of that would be if you ask a question like who do I talk to about WET Labs? Then that's probably going to get categorized as an employee expertise query. But if you say something like list employees with experience designing WET labs, that should be turned into a relationship query where we're going to list employees who have worked on projects where maybe the project description mentions a wet lab. Or if you've got a kind of project features column that would include wet Lab as an option, we would be using something like that. But one of the ones that I use is that keyword of list. If I want something that's going to be pulling a kind of comprehensive, filtered list from a directory, rather than just a kind of, here's a broad look across the Internet for something.

So I just want to build on. Everything Sean said is true. I think one of our core design goals with synthesis, though, is to try to prevent people from having to be able to be prompt engineers in order to get good results. So we are, we are. And this is kind of actually one of the reasons for knowledge agents also is, you know, we see people writing super pasting in super long prompts or writing super long prompts and then applying filters afterwards. And we just think there's ways to democratize more advanced AI usage, you know, without people having to become prompt engineers. And so that's both something we're trying to do in the AI search product itself, but also through knowledge agents. So we think that those agents will allow, you know, so some of that systems instruction or the job description that you have to write that's just this could allow like somebody who's a power user in your marketing team or a power user in your sustainability team to just level up everybody else because that they can embed all of that knowledge they have about how to use AI really well into that description and into the grounding knowledge. Okay, so Mohammed asked, are we introducing MCP for tools? So for folks that don't know mcp, that stands for Model Context Protocol, in layman's terms, it's kind of an API or a way to integrate with AI tools. I'm overly simplifying a little bit. And the direct action is we've been watching MCP very, very, very closely. We don't think it's ready for our use case specifically some of the things around permissioning to be used in an enterprise. There's other problems with it now, but it is a promising technology. So we are following it closely. We don't have any current plans to introduce mcp. There may be. There are other technologies like agent to agent from Google that are also emerging. So at some point there's going to be a standard, those things will mature a little bit more. And when it's there and we see enough demand in our community, we'll evaluate the right technology at the right time. So that's what we're thinking around that. Anything you wanted to add on mcp, Sean?

The thing I was going to say is that I think that may need to wait for some models to get a little bit better too. Because something that we have tried a couple of times is to see can we compress this query tree a little bit, take out some of these steps, just send all of these different tools, send all the project tools, all the employee tools to the LLM all at once and let it pick between them. It gets much less reliable if we do that. If we were adding on top of that tools that were coming in from in MCP integration, then I think that would be making things less reliable and repeatable overall.

Yeah, and I think inferred in Mohamed's question a little bit was the inverse too. So there's us using being able to take advantage of MCP tools in other platforms, but then would we open up the ability to query synthesis via mcp, you know, as well. And the answer is like kind of so Mohamed. That's correct, Chris. Yes. So yeah, it's something we're exploring and related question from Marcelo about do you provide an OpenAI for companies to create their own integrations? We don't currently, but it is something we have been tracking and scoping. You know, we're calling it kind of a search connector API at this point. I don't have a release date on that. It's not something we've seen a lot of demand for currently. Certainly not as high as the demand for advanced employee search, advanced project search, knowledge agents, you know, that kind of thing. But at some, at some point, if it becomes something that a lot of our clients want to take advantage of, we would certainly look into doing that. And if you want to Marcelo, send us examples of like what connections and what integrations you'd want to create. That would be super helpful for two reasons. One is we might build the native integration ourselves. Like there are some additional native integrations that we're considering. And number two, understanding the details of the platform and the use case would help us shape that. If we do decide to build the API, it would help us shape the API and make it the right way in the beginning point. Right. And so, Marcel, I get the question. So the open API is not using rlm, but to push data automatically and to make it available. That's exactly where I was going with that. So, for example, we have an integration with Zendesk, right? So that we can take their help center content, we can ingest it into synthesis, and then that content's all available to API, that Zendesk information is all searchable via AI search and synthesis. And so what the idea of a search connector API would be is to say, like, hey, you've got product X, you've got some firm wide knowledge that you want to be searchable inside synthesis. How could you push that in? And so that would be helpful for us to understand what systems you're thinking about and what they are and what the use cases are. So that could help us scope out again, do we just want to build that external integration natively ourselves? Or if we do, try to make it part of the API, help us build the right API in the first place. Cool. I think that's all of our questions. I appreciate you guys hanging on to the end. I apologize. We ran over a little bit. This is a new presentation for us, so we don't have all the kinks in the timing worked out, but we wanted to share the behind the scenes of how we're thinking about this, the decisions we're making. And again, if you're interested in advanced project search or knowledge agents, or you want to send us information about APIs and integrations, emailing us at support Knowledge Architecture is the right way to get it in. All right, thanks, Sean. Thanks, Susan. Behind the scenes.

Thanks, Chris.

Thanks, everybody. And we'll see you guys at the next one.