Synthesis Next Generation Search: AI-Powered Enterprise Search for AEC Firms

Hi, this is Chris with Knowledge Architecture and I'm here to share an update on our progress towards building synthesis. Next Generation Search Next Generation Search will bring AI powered enterprise search to synthesis. In this video I'll share our approach to both AI and enterprise search, but I want to start with why we're building next generation search. Ever since ChatGPT went mainstream, we've heard from several clients who immediately saw the benefits of applying generative AI to synthesis search. Today use search synthesis using keywords like jury duty when perhaps what you really want to know is how many hours does our company provide for jury duty service or what is our jury duty policy. And today we use keyword search to return a prioritized list of results when what generative AI and other recent advances in search technology make possible are answers which summarize useful information from multiple search results automatically and provide citations and links to the sources of those summarized answers. In this example I'm using a keyword search of work sharing where what I really want to ask is should I use revit, work sharing or central files and then get back answers which synthesize information from multiple sources and if I click on one of those sources in this example, a video, I'd like to be taken directly to the relevant passage in the video. Cool, right? Okay, now that you know at a high level what we're building and why we're building it, I want to spend some time talking about how we're building in talking with clients, friends, family and listening and reading online. I think this is the mental model that most people have of how AI works. So a search query would come into synthesis, it would go into an all knowing user facing large language model like ChatGPT, and then that large language model would generate answers. This is a bit overly simplified, but it is essentially how ChatGPT works. An end user facing large language model is not the search architecture we're using for synthesis. Next generation search for several reasons. So there are three core challenges with using end user facing LLMs for enterprise search. 1. Hallucinations LLMs are trained on vast data sets consisting of text from the Internet, books, articles and other sources. These data sets may contain inaccuracies, biases, or fictional content. An LLM learns patterns and information from this data without the ability to discern truth from fiction, which can lead to inaccurate or misleading answers which are commonly called hallucinations. 2. Freshness the LLM approach can also lead to outdated answers quickly, both in terms of holding on to information too long and not recognizing new content fast enough 3 permissions it is basically impossible to honor permissions in this approach. For example, if you had a private principals, shareholders or management team community in synthesis and we trained an LLM using that data set, all users would be able to potentially obtain answers from the LLM based on that private data set. This doesn't sound great. Chris, is there another way? I'm glad you asked. We've been hard at work over the last year researching, designing and now building a a better approach which overcomes the core challenges of end user facing LLMs. Let's take a look. The approach we're taking is generally referred to as Retrieval Augmented Generation or RAG for short. In this approach, a search query comes into synthesis. We execute what's called a vector search. More on this later, which retrieves passages of key text from the most relevant resources such as videos, pages, documents, projects, employees or posts. It relays those passages of text to a private large language model which summarizes those passages of text to provide answers with citations as well as additional search results. And at the same time, we also execute a keyword search which uses the synthesis search index to return traditional search results. We're using this hybrid approach for synthesis Next Generation search because there are many scenarios where keyword search is still the best technology for the task at hand, for example Employee or project name searches. So why is Retrieval Augmented generation better? To answer this question, let's return back to our list of challenges with end user facing LLMs. As a reminder, the three core challenges of using end user facing LLMs were hallucinations, freshness, and permissions. Let's take them one at a time. To counter hallucinations using Retrieval augmented generation, we ground synthesis search in your data, meaning we start by searching synthesis for the most relevant passages, pass those along to a private behind the scenes large language model, and require that large language model to only use content from the search which we passed along to generate its answers. To counter the freshness problem, we execute a real time search for each query, meaning that your most recent relevant and published content is passed along to that private behind the scenes LLM. And the risk of outdated or deleted content doesn't apply when we use Retrieval Augmented generation. And finally, with regards to permissions, because we're executing real time searches, this also means that the passages we pass along to the private behind the scenes large language model honor synthesis permissions. Nice. Now that we've seen how Retrieval augmented generation can improve AI powered search by grounding search in your data and executing real time and permission search, I'd like to introduce you to three additional concepts which are critical to understanding our approach to building synthesis next generation Search first, we'll talk about Vector search, where we'll explore how we find the most semantically relevant passages for a given search query. Second, we'll introduce our Community AI program which is focused on improving search performance by building AEC specific AI models and finally, Enterprise Search, where we'll talk about expanding the reach of synthesis search into more sources of high quality, firm wide knowledge. Let's start with vector search. There are two main approaches used when building a search keyword search, which looks for an exact match for a word or phrase, and vector search, which understands the context and semantic meaning of a search query. As an example, a keyword search query might be healthcare programming templates and the search results would include resources containing the exact phrase healthcare programming templates in the text of the content. Synonyms can be added manually via synthesis search optimization to improve the keyword search effectiveness. An example vector search query might be do we have any healthcare programming templates? In synthesis search would find resources semantically related to the word programming such as space planning, software development, scheduling, and so on, whether or not the word programming was actually used in the text of the content. The same concept will apply to both healthcare and templates, and the vector search will use the combined semantic meaning of healthcare programming templates to find the most relevant results regardless of the literal search query. Finally, using vector search synonyms are generally not required to be added manually. So how does this happen to overly simplify vector search? For the purposes of our discussion, think of a multi dimensional space with thousands of possible vectors. Words and phrases will be assigned locations. Literally numeric coordinates in that space will which places words with similar semantic meaning near each other. For example, programming might be near floor plan and space planning for one sense of the word. Programming might appear next to coding and software development for another sense of the word. And finally, programming might appear near scheduling and event planning for yet another sense of the word. Let's go back to retrieval augmented generation to see this in action. When a user executes a search query, we'll use natural language processing to determine the query intent. Then use the vector database to return the most relevant passages to a large language model to generate answers with citations. The final topic I want to touch on related to vector search is how the vector database knows how to find the most relevant passages to return to the large language model. The first step in the process of populating a vector database is for a user to upload or edit a synthesis resource Then we break the resource down into smaller chunks of text. For example, for synthesis pages we'll use elements like page headings or collapsible sections to split the page up into logical chunks. Next we use an embeddings model which understands that vector space we just talked about, analyzes the chunks of text and embeds vectors in them, and then we add those chunks into the right location in the vector database using their embedded vectors. So that's how the vector database knows how to find the most relevant passages for a given search query. The key to this whole operation is is the embeddings model because it establishes the vector space in which meaning is assigned. And because this is such a key piece of AI infrastructure, we are going to offer two different embeddings models to clients. First, our default embeddings model will be a generic open source model, and second, we'll offer an AEC specific embeddings model as a part of our Community AI program. Next on our agenda we'll take a closer look at our Community AI program and AEC specific models. In the pursuit of researching two use cases for AI next generation search and video captions, we discovered a unique opportunity to work with our AEC client community to achieve more relevant search results and better captioning. Thus, Community AI was born. Community AI is an opt in program which will empower Knowledge Architecture to create AEC specific AI infrastructure by training AI models on client content for the mutual benefit of participating firms. The first two Community AI projects will be an AEC embeddings model for more relevant search results and an AEC transcription model for more accurate video capture. By default, clients are opted out of the Community AI program and will use the generic embeddings and transcription models. Clients will need to explicitly opt in to join the Community AI program and get access to the AEC specific models. To build our AEC models, we'll start by extracting AEC specific terms, phrases and context from synthesis content at participating firms, for example Roof Cone Flashing Details, Deluge Fire Suppression sprinkler system, Deltec, Vantagepoint, Tekla structures, CEQA, NEPA and AIA 2030 which will allow us to fine tune our AEC embeddings model for vector search as well as fine tune our AEC transcription model for video captioning. Keep in mind these models are behind the scenes infrastructure used for data processing. Terms are de identified on their way into the models and at no point would end users be able to interact with them or ask them questions. In talking with clients I have found it very helpful to think of them as crowdsourced AEC specific dictionaries. If we go back to the diagram I shared earlier explaining vector search basics, you can see how powerful of an impact building an AEC specific vector database could have on your employee's search experience when you consider that a vector database containing semantically related concepts around facades or sustainability or building information modeling would help folks find what they're looking for even if they don't use the correct terms in their search queries. Okay, so we covered retrieval, Augmented Generation, the Vector Search and Community AI the last stop on our journey is talking about Enterprise search. Our vision for Synthesis search is to be able to successfully answer questions using your firm's highest quality sources of firm wide knowledge, regardless of whether that knowledge lives in synthesis or not. Because of our existing integrations with AEC software, our exclusive focus on AEC firms, and commitment to building AEC specific AI infrastructure, we think we're in a unique position to provide the best enterprise search experience possible for AEC firms. Here's what we're doing in version one, all of your Synthesis Intranet content and once released, all of your Synthesis LMS content as well will be available to Synthesis Next Generation Search as well as your directory and profile data synced from your ERP and CRM systems, Open Asset Images, newforma contacts, and Zendesk Help center articles. We built the Zendesk Search Connector last year because we saw an opportunity to give our clients the best of both worlds to allow them to create essential IT or design technology content like documentation tips or how to's in their Zendesk Help Centers so that those articles appear when their end users go to create new tickets, but to also make those same articles available via synthesis Search. So here's an example of a Zendesk Help center article being returned via a synthesis keyword search. This is possible in synthesis today. Here's an example showing Synthesis Next Generation Search, citing a Zendesk article as a top source for the AI generated search summary. So you might be thinking, what other firm wide knowledge sources would our firm like to make available to Synthesis Next Generation search? Good question. While we're currently focused on delivering version one of Next Generation Search, now we'd also like to start eliciting feedback from the community on the best way to expand the reach of Synthesis Enterprise Search into more high quality sources of firm wide knowledge in future versions. Here are some of the ideas we've heard most consistently from the community so far. First, for Knowledge Architecture to build more native search connectors such as Microsoft Teams to make messages and documents from public teams channels available to Synthesis Next Generation Search as well as freshworks to make knowledge based articles available to Synthesis Next Generation Search. Outside of Zendesk, freshworks is the most common help center software used by our community and we've had numerous requests for a search connector recently. In addition to more native search connectors, we're also exploring a Search Connector API which would allow clients to build their own integrations with Synthesis Search. We'll continue adding native search integrations, but we can't cover them all and we want to provide tools for our community to build search connectors to other third party software databases and custom applications. Finally, we're exploring the ability to index public websites via Synthesis Next Generation Search to help augment your search results with high quality and high priority content from trusted sources on the web. Clients have shared two main use cases for a public website indexer, first to index their own websites since it has high quality content which doesn't necessarily live in synthesis. And the second use case is to index the help centers of trusted software partners such as Autodesk, Adobe Open Asset or McNeil to help augment the company specific help and training content available on synthesis. As I mentioned, these are all future explorations and we'd love to hear your feedback on these ideas or any others you might have for expanding the reach of Synthesis Next Generation Search. We've covered a lot of ground together and introduced a lot of new concepts. I hope you're as excited as we are about Synthesis Next Generation Search and we'll keep you updated as we continue to make progress. To that end, one thing I haven't spoken to is timing. Our goal is to have a public beta of next generation search version one available in 2024. Because this is such new technology, we expect Next Generation Search to have a pretty long beta and we'd like to provide access to as many clients as we responsibly can as soon as we can to help us improve the product quickly. So that's it for our deep dive into Synthesis Next Generation Search. Please feel free to reach out with any comments, ideas or questions and happy searching.