Avoiding AI Hallucinations in AEC Knowledge Management With Synthesis

So how do you deal with hallucinations? Do you have any. Any times that it invent people that don't exist or just provide the wrong information like ChatGPT sometimes does?

Yeah, it's a great question. One of the reasons this retrieval augmented generation architecture was developed was specifically to counter hallucinations. So because we're rooting in your data, that does a lot to eliminate hallucinations. Because one of the problems with training models is they become out of date because they're just. They're only as accurate as the last time they were trained. And so I could add a page to synthesis right now as we're talking, do a search and that page can get referenced in AI search because we do the search first before we do the generation. So that really helps with hallucinations. Now, if you have inaccurate information in your content, somebody might consider that hallucination. In my view, that's not a hallucination. That's just a wrong answer based on inaccurate content. The other thing that we did have to train synthesis on how to handle though, is if this post had cited ada, but ADA was no longer with the firm. We had to teach synthesis to understand and check the employee directory first. And if she's no longer with the firm, include that context to say that she's a former employee. But generally the hallucination rate has been pretty low. Sometimes it gets confused about what your question is, but the models keep improving underneath our product and that has really become less and less of a thing and.

Yeah, but often the hallucinations are. When there's. AI cannot find good information, so it just makes stuff up. Right. So what happens if you search for something and there's just not good information about it? Will it say, sorry, we don't have anything to offer?

All right, I'm going to try something like this. So why is Steph Curry the best player ever? We're going to search the Internet across all the content and it cannot be answered.

Nice. Yeah. ChatGPT never says that. Right?

Never says it's. I don't know. So our product is a little more humble and it will tell you it doesn't know things.

Yeah, yeah, it's pretty funny. I like to use ChatGPT, especially if you try to use it for the. With the Revit API, for example. It's very confident. Here's the code that you should use.

Right.

And obviously it doesn't work. Sometimes you do back and forth and it ends up working, but it never tells you, sorry, I don't have enough information to accomplish.

It wants to be really helpful, I guess, if we want to be. Well, like, it just really, you know, it really wants to help.