This excerpt (early 2025 segment) describes the team’s transition from knowledge-base search toward advanced employee search. User feedback created a knowledge "flywheel" that revealed strong demand to find people (e.g., employees with wetland delineation experience, licensed PEs by state). Earlier retrieval architectures (keyword RAG → hybrid RAG) could surface the "best" passages but not reliably return "all" matching employees, so the team developed an agentic RAG approach: categorize queries, constrain LLM decisions with deterministic query trees, and provide tools (employee directory access, intranet) so the agent can return exhaustive, citeable employee search results alongside knowledge answers.
Chapters
So then part two, we moved into advanced employee search. And this brings us into early 2025. Our goal with version 1.1 was connecting people to people. And so this I mentioned earlier, when you submit feedback, I said, I'm going to tell you where it goes later. So it goes two places. First in, it goes to the intranet team, the knowledge management team, whatever you want to call it, they see all that feedback that end users are submitting. So when somebody searches, that feedback gets shared. You can improve your content. It could be that there was knowledge that was missing, something could be outdated, it could be confusingly written or duplicative, whatever it is. Like that feedback helps you improve your content, therefore you're improving your search. People are using it more, they're sharing more feedback, you're improving your content. You get into this really great virtuous cycle, or this great knowledge flywheel, as we like to call it. Also, when you submit that feedback, it comes to knowledge architecture. So this allows us to see patterns happening over queries. We start understanding where we need to make improvements in the software. And so that goes back. So as people are having better search experiences because the technology got better, now they're doing more searches, they're sharing their feedback. It is a very great symbiotic, dual flywheel, virtuous cycle of improvement that we've been on with our clients for the last year using the product. So out of that feedback, we start learning that people are trying to do queries like this. Who has experience with wetland delineation? And based on the available information on the Internet, it said, there's no employees identified with experience in this area. And I rate it and say, like, that's not true, that I know we have several employees who've worked on these projects and they're in the employee directory. Why aren't you finding it? Synthesis? Good question. So we in fact got lots of feedback like that. This is just464, and this is an early screenshot I took when we decided to start working on this project. And so you can see people are looking for all the employees in their Chicago office or who's a notary or who's on the board or who has experience with Wetland delineations. Like, we saw so much of this feedback and it was so clearly the number one thing people wanted to do. Once version 1.0 came out for the knowledge base search, the next big thing they wanted to do was search for employees. So we realized for the Jaws fans out there that we were going to need a bigger Boat and that boat. Well, let me tell you what was missing for the boat. Why our boat was too small. To answer those employee questions, right, we had to be able to give an answer that included the best things as well as all the things. So the best things is if, if I'm looking for the WI FI password, or if I'm looking for a 401k match, I want the best content to get me the answer to that question. But if I'm looking for who has experience with wetland delineation, or who's in our Chicago office, or who's a licensed PE in Washington, I don't want three of them. I want all of them. And that was something that that current hybrid RAG approach I showed you before just wasn't designed to do at all. So this is it, right? This is that hybrid workflow. You can see we're trying to retrieve the key passages, the best things. Like it's literally architected to not solve the all the things problem. So we also wanted to make better use of the employee directory and employee blocks. We have good employee data based on integrations with deltec and UNET and Open Asset and so on, as well as these employee blocks on pages. So as an example of the employee directory, you can see that we've got department discipline office, as well as a bunch of other data points. We should be able to take advantage of these in synthesis. AI search is what we were saying in the spring of this year. This is an employee profile. Again, you can see these fields about M rich information on a professional bio, a personal bio, skills licenses, certifications, all these things. In addition, in this example that Sean showed you before on WET Labs, we. We've got Ann Johnson here on the right as that employee block. We wanted to be able to factor that into a question on maybe like, who should I contact about WET Labs? Right. In the first version, we weren't able to answer that question very well. Okay, so what was missing from our boat is now. So we started with keyword rag. We then went to hybrid rag. Version 3.1.1 was agentic rag. So Sean's going to talk about our first swing at this, and then I'll kind of talk about the revision, then we'll come back.
Yeah. So there's a truism that it's good for a computer programmer to be a little bit lazy. So in true computer programmer fashion, I thought we would just see if the robot can do it for us. So we set up this structure where we would send the question to the robot to The LLM let it retrieve. Generate a query to retrieve results, look at those results, and then if it thought they were sufficient, then write a response. With this example of who's our revit guru? We would send it that question and then it would first try searching for revit and get too many results, or try searching for revit guru, and if we don't mention the word guru, then it doesn't get enough results and then eventually get its way to something like revit expert, and then that would sometimes end up with it getting the right answer. And the problem that we saw with this was that it was really inconsistent when it worked. It felt amazing. It seemed like this whole AI thing is really going to work. We can just let the chatbot solve these problems for us. But that happened maybe 60% of the time, and that's not good enough for a search use case. So we really wanted to figure out a way to improve that consistency by constraining the scope that the LLM operated within so that we could provide more repeatability and make sure that we're generating good, relevant search summary results all of the time and not just 60% of the time.
Right. Which brings us to, again, as little AI as possible. So we realized we have let this swing too far in the probabilistic direction, or we have given too much responsibility to AI, which it's not ready to handle. So we needed to claw some of that back and make sure that we're giving it good guardrails and sending it down the right path. Right. And I. So we'll get into agentic rag. So agents has been the term of the year in 2025. Everyone's got a different definition. For me, it's pretty simple. AI comes with reasoning capability, right? It's got some intelligence, it's artificial, but it does have intelligence. So it can make decisions. What we do as both software providers, but then also you as users of the technology, is you provide it with the grounding knowledge it needs to make good decisions or answer questions. We also need to provide it with context. Sean was talking about context engineering. So agents need reasoning knowledge, context, and they also need tools. So tools let them do things like access the employee directory, for example, in a way that lets them get all the things right. And so we need to continually build tools to help AI do its job. So this combination of reasoning knowledge, context as tools is to me, what makes for an agent that can make good decisions. So how does that manifest in a synthesis? So now when a query comes in, this one's an employee related query who can help me with a drone survey? What we do first is we ask the AI to categorize the query. It's a very simple question. Is this more about a general knowledge base query or does this sound more like it's a question about an employee? Like it's a very simple question. And if it answers general, we then have it use that hybrid rag approach we saw before where it's trying to find the best stuff, right. To generate its answer. If it says this feels like an employee query, we then ask it to further categorize. Okay, so is it more like a search for an employee name? Is it more like they're searching on role or title? Is it more like they're looking for who has expertise with this thing? Or is it a straight up filter in the directory? So licensed PEs in Washington. And so based on its question, it will use different tools. It could then even go out to the intranet to supplement its knowledge, if need be, to then generate an answer with citations. Right? So this is agentic because we're giving it tools, we're giving it context it needs to make decisions, access to knowledge. And it's also what we call a query tree. So we've built this tree structure, a deterministic tree structure to help AI navigate and use its probabilistic capabilities and more effectively. But we limit the scope of what we're asking it to do. We're basically asking it to make judgments about things. And we give it also assistive language to help it understand how to make those decisions. Now we can ask questions like who can help me plan a flexible wet lab? And we're back to that Ann Johnson example, like on that Wet Labs page where we saw her as the employee contact, or who are our PEs in Washington? We can answer that question. And in the way that we designed the ui, we're also providing explainability. So you can see the reason we said it was six is because in the directory we took licenses equals professional engineer and license by state equals Washington. Or who is our mass timber expert? Right. And we're able to in this case cite Internet content. So it could be a video, is the reason why. Or who can help me with Dynamo scripting. Then this might come from an employee profile. So this might be in her. I'm a resource for her skills, perhaps. So we've got all this diverse information, some of it in the structured directory like the employee directory, and some of it unstructured in pages and posts. And so the goal for this version the advanced employee search was to take advantage of all of that knowledge and get connect people to the right people revit container files. So one thing that we wanted to do here, so this is a knowledge based query, which is what we went after in the beginning of synthesis. But what we always wanted to do is to do both if we could. Right? So in this example, we're answering your question, right? We're describing what a revit container file is, but at the bottom, we're also suggesting that you can contact Brianne Walton, who leads this area. So in the best thinking back to my definition of knowledge management, where we want to connect people to knowledge and connect people to people, ideally we're doing both. And this is an example of when possible, we we do both for those knowledge base queries.
