Kevin Surace explains how they test apps in a digital twin called GENI. They use a rule-based code generator that takes instructions from AI but produces exact code from fixed rules, sidestepping hallucinations. Additional patented checks and offline knowledge make sure tests stay accurate.
So you're creating an interesting sort of, sort of, you know, analogous world. Right. But I'm wondering how much, how many of the analogies roll over from the common world of AI we're used to today to this highly specific, highly engineered, very smart environment of a digital twin and genie. So, for instance, if I'm looking at testing the app against these large variable rows of data, do I worry about stuff like hallucinations? Do I worry about.
Great question. So rows of data or not hallucinations can be a problem with a transformer. We use the term hallucinations because we're trying to anthropomorphize the fact that the math spit out the best it knew how to spit something out. And when it put word after word after word together, the resulting sentence may be totally made up. And you remember the job of a large language model is take a word and put a word after it and put a word after it based on the probability of that word coming after the prior word. Correct everything it's ever learned. And it could end up putting a whole bunch of things together that in fact don't make any sense at all. I mean, the phrase makes sense, but it's not accurate. Right. So we got around that actually quite well by using a code generator. So we built our own code generator. And what we're doing is we're giving instructions to the code generator, but the code generator writes exact code. So we don't use an LLM to write code, we don't use AI to write code. We use AI to tell the code generator what it needs to act on. And the code generator uses rules to act on those. And a rule based code generator, which is kind of like a compiler, if you will, it works. We don't question whether our compilers work. We know they work. Right? They work. Why? Because based on a set of rules, and the rules are fixed, that's how we got around that. I think it's a brilliant solution to a very complex problem. And so the thing can't hallucinate. That's the bottom line. There's another set of checks and balances that we're not talking about in detail publicly right now, but they're in some upcoming patents and they're very powerful. But I can say when you have a digital twin offline, you have a lot more knowledge of the application than you can get live. And by using the knowledge from the digital twin, we can really represent 100% that we have mimicked what you've asked us to do in the test case, and that might blow people's minds a bit because they're used for this amorphous stuff rather than accurate, but, yeah, but again, if you've got a set of models, by the way, you can do voting, you can do judging, you can do all kinds of things to make sure that you got over here must be correct.
