A short conversational interview where Benn Stancil argues that AI (especially large language models) will produce a "weird" future for data and analytics because they cause discontinuous leaps in how people work—analogous to being teleported to a new world (pre‑iPhone vs post‑iPhone examples). He emphasizes that LLMs behave differently from traditional deterministic software: they are creative, human‑like in their unpredictability, and often resist strict instruction. He cites the idea that so‑called "hallucination" can be reframed as creativity, and that treating LLMs as perfect obedient tools misunderstands their character.
You know, one thing that we chatted about behind the scenes when we were talking about this episode is that resonated with me a lot, is that you talked about AI assisted analytics workflow in the future being weird. And I think the word weird does a better job than most other adjectives describing the potential world we're heading into. So maybe you've set the stage. What do you mean when you say the future of AI and data will be weird?
Well, I said two days. I get one is generally to me the way, like, tech, and this is not a novel thought necessarily, but the way that, like, technology and stuff changes, like big technological shifts change how people do things. It's like teleportation in some ways that you. We don't like, sort of walk slowly towards it where we can see what's coming and it's like, oh, we see it in the distance and we get closer to it. It's like a big thing comes out and you sort of get teleported to a new place where you have no idea what's on the other side of that thing until you get there. Like, you, if you like, snap your fingers, like, guess what? You're going to teleport somewhere. You're like, I have no idea where. I could, I could end up anywhere. Like, it's impossible to imagine the next five steps when you do that. And I think there's like a version of that where these big changes happen. And once you have some new kind of foundational piece of technology, a bunch of new stuff happens with it that you wouldn't really have been able to imagine even existing before. Like, it's, it's, you know, there's, there's, it's very hard to explain if you, if you were pre iPhone, like, what people are doing with TikTok, like, like the, the, the sort of, that stuff, it's like, doesn't really even make sense until you, like, imagine, oh, you have these phones, you can like, do all these sorts of crazy things with them. You know, you have like all these sort of location services built into them. There's all these other, like, capabilities. Like, you have to kind of have the phone first before you can even imagine, like a thing like, like TikTok or a thing like Instagram or whatever. And I think AI is to some extent that. Where, like, whatever happens to it, it's going to be. The people see it and they're like, oh, great, this is a chatbot. Okay, we're going to go build a bunch of chatbots. It's like, nah, probably not. Like, that's obviously a first thing. When you first build a phone, a bunch of people build sort of the easy extension of what the world was like before. But once you start to actually play with it, you realize, oh, this thing can do all these sort of unexpected and weird things or sort of have our, our mind is opened up to a bunch of ideas that we wouldn't have really thought of until we got to this point. So generally to me that's, that's like it's, it's going to be weird because it's, it's a, it's a big foundational shift that we can't sort of just extrapolate the, the previous what we've been doing. It's like, it's like a discontinuity. The other thing to me about them specifically is AI and LLM specifically are a very weird kind of technology where like they're very human in that a lot of the things that humans do that like they are pretty creative, they aren't very good at taking explicit directions and doing exactly what you want them to. They do unexpected things. They like, you can treat them almost like humans where like consistently. To me the thing that is surprising is when you want them to do something better, you do the same thing you would with a person where you like, you just give it more information, you kind of explain it more stuff, you ask it for more specific results. Like you tell it to give it feedback and it kind of learns. Like there's a bunch of weird stuff in that. That's like a pretty weird. It's not a computer. It's a computer obviously, but it's not like a computer that thinks the way that, that we. I've always thought of computers thinking. And so to me, like what happens to that? I have no idea. Like there was this, this was the Ilya, the, the. Yeah.
Chief scientist.
He is currently now at OpenAI.
Yeah.
Had a thing about like people complaining about, about hallucination and in LLMs and he's kind of like, look, hallucination is the point. Like hallucination is. It's not hallucination, it's creativity. And that the problem is not the problem. Like he was comparing LLMs to search and he was saying the problem with search is it has a. It does. It has no creativity. Like there is. You search for something, you're hearing the same thing all the time. Like it is not a creative thing. But LLMs are create like that. That is sort of the benefit of it. And I don't think we've grasped that yet. I think we've, like, view them as how do we make them perfect little robots that do exactly the task we want? And it's like, they're not really that. They're kind of this weird, creative robot. And that's kind of a crazy thing to think about. But, like, what happens? That I don't know, but we're not used to, I think having a tool that is more creative than we are in a lot of ways, but is not good at following directions.
