Benn Stancil - 2024 Predictions, GenAI and Product Development, etc

Happy Monday. What's up, guys?

Happy Monday, Joe. Welcome, Ben.

Ben, how's it going?

Good, good, good.

Awesome. Good to have you again. Yeah. So turns out you guys are actually in the same neighborhood or ish, I guess. But it's good to. Good to have you all dialing in from Brooklyn. How was everybody's weekend?

It was great. So of course we were at Data Day Texas, which is a party that masquerades as a data conference, basically.

Yes. Not a masquerade party, unfortunately. But yeah, it was a fun time. So good to see everybody in Austin, Texas, Shout out to Lynn Bender and Alex Wall, who, you know, put together, I think one of the best data conferences on the planet, not just the US So it's really cool. Just a lot of hitters in the space getting together over drinks and talks to hang out for a Saturday. So a lot of fun.

So Ben, what are your 2024 predictions for data parties? Are we going to have more of them or is the funding.

For data parties? Yeah, they're probably not going to be as good as they were in like, say 2021. I don't know.

There was.

This was back, way back when that I remember we. One of the first jobs I had in tech was like a startup kind of 2012 or something like that. And we had a Christmas party, like a holiday party that was. Had like Cirque du Soleil performers and like a full New Orleans brass band and stuff. And it was definitely one of those moments where it's like, okay, this is. There is no way that this continues. So it feels like we're a little bit in that moment where, you know, the, the four story party with three different whiskey bars for, you know, whatever flavor you prefer was. Is not a thing that we're gonna have anymore at these data conferences.

What kind of parties do you think we're gonna have going forward?

We're just gonna be jealous of the Gen AI people's parties because they're the ones with all the money. We're gonna become like the, the new, like, I. T. Folks where there's like, the cool kids are all having their crazy Gen AI parties and we're, I don't know, wearing our khaki pants and our polo shirts and that you got.

From Mervyn's or someplace. Yeah.

Talking about how we're. We're the ones who are really thinking about value, unlike those, those kids that need to get off our lawn.

Yeah.

Well, it's funny actually, in the Bay Area, right. I think there's a. There's Gen AI meetups every night. It seems even the kind of ML Ops meetups have gone to the old school. Like nobody, you know, there's like, who wants to care about that. But you know, yeah, I think it was at shack 15 or whatever the fancy place is in the pier, like events every night there for Gen AI meetups. I mean you could probably hit 10 or more gen AI meetups a night in the Bay Area. And they're all awesome with great food.

And so they have a lot of money. There is like, as a slightly more substantive point, there is something to me sort of interesting in that, that, that as it relates to data people, which is like Gen AI is not a data thing. Like, like it is in the sense that obviously like the technology is built on data stuff. But like largely if you're building stuff with Gen AI tools, you're not doing anything. It's not related to data at all. Like, like a data scientist and someone who builds like Gen AI products are not the same person at all. But because like AI has always been kind of a data thing and like machine learning and AI are kind of adjacent and machine learning and data science are kind of adjacent and all these sorts of things, like you end up with this kind of weird world where like data people are sort of seen as like, oh, AI is kind of your thing too, but it's like not. And so I do think there'd be a lot of like data people that try to kind of make that shift of being like, oh, I'm a data scientist, I'm actually an AI person. It's like, but you've never thought a thing about like an LLM or about especially like, you know, mid journey type of stuff. You don't know how any of this stuff works. You make regressions to figure out if leads are good or not. But because there's like just this naming thing that's an overlap, I think there's gonna be a lot of people who sort of try to make that migration. Like it is, it is just the name like gen AI and, and sort of AI and what it was five, 10 years ago is like basically the only thing is AI could have called gen AI something different, but because it's called AI, we kind of were like, oh, it's a, it's, it's similar to what data scientists do, even though most data scientists are, you know, making recommendation algorithms for stuff or whatever.

Yeah, that's interesting. I, I think it's comparable to what happened in like the Hadoop era where you kind of had some data people who just stuck with their teradata, their Oracle and then other people just went all in on Hadoop and, and like totally reskilled and said hey, now I'm a big data person. In fact they often said that before they'd really learned the skills they just like typed in a couple of commands and now they're big data people.

Was that really a step forward though? I don't.

Well, in retrospect, no. Right. The question is are we in the same kind of cycle here? We've been talking about hype cycles here. So yeah.

Like is Genai the next Hadoop?

I. That doesn't seem quite right to me but I think we are, we are maybe at peak hype. Unless the hype just continues to grow through 2024.

I don't know what would cause it to keep growing.

That is a good question. So is it, is it going to be like the late 90s where the Internet cycle just like ramps up, ramps up for years and years and then all of a sudden it drops off all at once. Like you just get this like building public social excitement, massive investment. We're seeing that in the stock market certainly where stocks like Nvidia, what is it the market cap is now 1.5 trillion or 1.6 trillion the last time I checked. And the question is how long is that sustainable and is that bubble going to pop at some point, say later in this year?

One thing that is somewhat striking to me, I have no idea if this is something you could actually say is similar to the, the history of the Internet too. But I mean like, but Chat GPT is now a year and change old. That was like when everybody suddenly lost their minds about this stuff. How many people have actually built sort of meaningful products on top of Chin AI? Like not many. There are, there are a couple things like image stuff that seems like it's good. There's like the mid journeys and the runways and stuff like that which seems like okay, there's. Is there business value in that? Yeah, I probably, I assume but a lot of the like LLM Chat GPT bot stuff is like what are the big success stories so far? Copilot co pilot makes a lot of money I think but like that's kind of it, you know, like, like. So I work at thought spot. ThoughtSpot has some tools like this that are pretty good for, for bi but they existed like thoughtspot had built a lot of this stuff prior to the Chat GPT wave. But like in terms of huge like world changing technologies Like, I don't know, engineers have better autocomplete. Like that's what we've got. Like, it's, it's, it is a little bit striking the degree to which we all got ramped up about this stuff and then nothing is really that different. Now obviously that may change, but, but it, it's like it's not, it hasn't been like this amazing immediate thing yet. It's made a lot of nice demos. Seems crazy, but in a lot of.

Demos, I mean, remember walking around big day to London when we were hanging out actually back in September last year, and it seemed like every vendor had an AI demo, whether or not they were actually an AI company or not. It's like, well, check out our new large language model interface. And so it occurred to me, walking around the conference hall, I was like, okay, so I don't think this will necessarily increase the tam, the total addressable market for data tools. But it's so different than when BI tools included things like charts. It just becomes another interface at the end of the day. So I'm like, okay, so I guess you're just competing on including this at this point. But I don't see it as being a differentiating factor in that sense. Right. It's. Yeah. Interesting.

This is to like the 2024 predictions thing. We just did this.

Yeah.

Speaking of thoughts about did this like, you know, sort of annual prediction white paper deal. And one of those things on that for me was like in this chat bot stuff that that was the default that we kind of everybody jumped to was Chat GPT was a chat bot. Chat GPT was very popular. We need to build a chatbot. And so, so many products tacked on chatbots into places where it's like the new interface should be a chatbot. And you're like, oh, we're a tool that, you know, notion needs a chat bot. Where Zendesk, we need a chat bot. Where internal help sites at McKinsey, we need a chat bot. And it seems a little weird to me. It's like that's not probably the right interface for this stuff. It's just like that was the first popular thing and so we all tried to build a version of it. And so I do think there'll probably be a phase where like we get a bunch of these weird tacked on chatbots that actually make products not better, like they're probably like obnoxious. But that to me is like, ultimately if this does actually become a really big thing, it's that we figure out Other ways to integrate that kind of technology, other than just making it like a chat bot that talks to the ui, which seems to be the default now, is like instead of controlling this by typing code or by typing stuff and pushing buttons, let's make it a chatbot. Okay. Like, you could imagine doing a lot more interesting things if you have some technology that can take kind of ambiguous commands and sort of infer what you mean, which is more or less what it does. And so I think there'd be like more interesting ways that people start building LLMs into tools where it's less like you don't even notice necessarily there's an LLM in it. I think that's when it starts to become interesting. Like when it's so obvious, guess what? We have a chatbot, that it's a chat GPT powered chatbot. And you like immediately know that's what it is. I don't think that's that interesting. When you just use something that just seems to do stuff that works really well, but you don't even really know how. That's, to me, the point when it gets like these things are actually making products way better rather than these kind of weird glommed on features.

I agree.

And Matt was mentioning the Internet before. I mean, I was developing web pages way back in the day. I made sure I was testing Mosaic when it first came out. So I've seen the rise of stuff. Right. And it's like HTML back then used to have, I think a blink function.

Right.

See, Remember those obnoxious geocities pages with the flashing everything? Yes. This kind of reminds me where LLMs and generative AI is right now, where it's kind of, it's cool, but it all looks the same and it's kind of obnoxious in that way. And it's very primitive. Right. I think to your point, when you don't. But it's not like the Internet went away. We're on it right now talking and I think. But we don't acknowledge it. Like, wow, look at the power of the Internet. It's, it's amazing. We're here. It's amazing. Thomas Edison, you know, really democratized electricity back in the day too. We, we don't fathom at this stuff anymore. It's just sort of taken for granted. And I think you're absolutely right, Ben. When we stop acknowledging the AI part because it's so embedded in everything, I think, well, your phone's been doing this for ages.

I mean, look, I mean, that is true.

Yeah, we don't marvel the AI powered camera. We don't marvel at that, even though the cameras in the iPhone are pretty ridiculous, you know, so.

And I think another good parallel with the late 90s is that we were already figuring out what the utility of the Internet was going to be. So various companies, almost as a novelty, started coming out with online ordering and virtually no one used it for years. It was just like, oh, that's cool that this company has online ordering for pizza, but why would I ever use that? And now where are we? How often do you pick up the phone to order pizza these days? You know? But I think it's something similar right now where everyone has to get this like these LLMs and in their products and they're just not really being that useful to Ben's point. But maybe in 10 years they'll, you know, that's going to completely take over.

The geodes thing I like as like sort of the MySpace version of this where everybody has this like aki page that's like cool. I guess we thought it was. It's like, let's play some, you know, 182. When I on the page, it's awesome, but. But it's like not useful. There's a lot of in there that. Yeah, you figure out. It's like when the LLM infrastructure instead of a feature, then we'll be like, oh, this thing really does some crazy stuff. Because like I think the phone piece is interesting where it's like there's probably tons of things that phones do that you have no idea what they're actually doing that it seems like, like roughly it out. Like I've never. Obviously nobody's ever used one. But the Apple Vision thing, like it seems like you're just looking through a piece of glass, but you're actually seeing a screen. And the screen's probably doing a ton of insane things as how it presents it. It is not just a camera broadcasting into your eyes. It's like taking a tons of images from a thousand different cameras, like twisting them all together in some weird way and then stitching them into something that actually like makes sense. And you feel like you're just looking at the world, but you're looking at nothing that's the world at all. You're looking at like this enormously complicated representation of the world. And I could see there being something where LLMs kind of become that kind of interface through which we like interact with computers, where it feels like we're just interacting with the raw thing, but in reality we've been sort of like it's been stitched together for us with all this sort of stuff in some crazy way. And I have no idea what that looks like. Obviously like technically they're not that capable yet and they're way too slow for anything like that and all that kind of thing. But I don't know, you could see that being kind of a. Yeah, almost invisible interface through which everything gets run that we don't feel it but it makes it like all just easier to use.

Way back in the day I always thought that the Internet would feel. I mean I remember like reading old Wired magazines in the early 90s about like VR and how that was going to be the interface that everyone used. And so it's been sort of this elusive thing. But I always envisioned or I always wanted a world where I didn't have to stare at a screen or I just sort of go throughout my daily business and just interact with the environment and just does things. It's sort of like a mix of like 1985 and I guess 2085 but I don't know. Back to gen AI and product development, that's one of the things we want to touch on. Like how do you think generative AI impacts product development?

I mean like my. The first thing is I think there's a lot like this basically this chat bot hype where a lot of people are, to your point about, about Big Data London. It became necessary for companies to be able to say what they're doing with it. Like a lot of you're, if you're, if you're selling an enterprise product and speaking this mostly from an enterprise perspective, I don't know if consumer perspective is like this or not. I don't really deal with those sorts of products that much. But if you're selling enterprise tech like your buyers are going to be like what are you doing with Gen AI? Like we run it, we talk about it all the time. Everybody's talking about it like we need things to say that we're doing this and it's just like a lot of interest in it. And so a lot of people I think immediately rush to just tack stuff on to be able to say here's our Gen AI roadmap. Look at the stuff we're doing. The obvious stuff was again chatbots or something that's roughly that it could be like type of thing and you'll get some better version of autocomplete or whatever. I don't know. And so I think like there's a lot of kind of rushed features out. To be able to say we have an AI product that doesn't really make sense is like a better experience. My guess is that we'll go through two sort of cycles of this. There's like one, there'll be just like the tacked on stuff. The second cycle will be people trying to make that experience a lot better where it's still clearly an AI product. You'll still have the little AI icon with the little stars that now has become what AI means. I guess I don't know why that.

Is.

But the inner experience will make a lot more sense. So like Mo, we are at Mode, we have worked on a version of this where like Mode is basically a tool. You can write SQL in it. The way that we've tried to do it is what if you can kind of outline a SQL query? Like the idea is if you're a designer, maybe you want to like draw a sketch of what you want to design and then like you got to go through a lot of work to fill out all like the tedious pixel perfect stuff. There's a flavor of that in writing a SQL query where it's like, I don't want to say as an analyst, I don't want to ask a question of like some business question. I want to say like, how many coats did we sell in California last month? Or whatever. I'm asking more like difficult questions than that. I want to basically say like, here's the query I want to write, but just like sketch it out and fill in all the details. So we're trying to do stuff like that, which is like a sort of step beyond a chatbot. But it's still very clearly an AI powered thing. So I think we'll go through a phase of that where like there's genuinely start to be some, some real valuable products. And then the third phase will basically be this thing where the AI disappears, where the button with the little twinkly thing disappears and the thing just does what you want it to do. You know, it's like the magic eraser thing on the phones or whatever. Or like you take the picture and it just automatically kind of corrects things for you somehow. Or you know, like phones, I think to your point about how they like, you know, take a picture, it's going to automatically do a ton of stuff to like adjust the image to make it work. Stuff like that, where there's just like these little tweaks and stuff that just make things easier to work with. But I think we go through this phase of kind of ugliness where we are tacking on a lot of stuff just for the sake of. We got to have like, we got to have our booth at Big Data London. Say how we're an AI powered data contracts company. I don't know. And, and so like, all right, we're gonna give you some AI assistant that takes the code you write, stuffs it through chat GPT and sticks out something better that says like, do you want to rewrite it like this? And you're like, no, but okay.

That's hilarious. Yeah. And it's, yeah, I think it's, it's, it's kludgy too because I mean, as an example, talk about demos at conferences over the weekend. One, one vendor who I won't name they, at 3am the night before the conference, somebody got a hold of their open AI key and started racking up charges.

Oh.

So they had to shut it off and now they had like a local, kind of like clunky local demo that wasn't nearly as good as making a open AI call. So that also happens. So.

And like, yeah, and there's obviously a whole bunch of questions there about, about like costs and security and how much of this do you know? Like when you actually do this for real? How much can we stuff stuff into OpenAI or you know, Google, Bard, Gemini, whatever, those sorts of things? I don't know. That, that to me is a, is a sort of messier question that if it becomes really valuable, I don't think anybody will care. If it doesn't, then that'll be a real sticking point for a lot of folks of like, we don't actually want to ship stuff off to, to Microsoft. As a somewhat tangent, I like security. I think, I think that's how basically all security stuff works, especially in the data world, is like, there are no hard security requirements basically for anybody. If you do something valuable enough, people will be cool with it. And the question then is just like, can the open. Can the AI stuff be valuable enough that people are like, yeah, fine, ship it off to Microsoft, I don't care. Or is it, it's kind of useful. So I'm going to get like real stickler about how we actually develop the stuff and all that kind of thing.

Here's another question. What do you think about the liability of having AI in products? I think we've, we've talked a lot about the utility, which so far is kind of limited, but there is this hidden liability question. I kind of think at some point in the near Future we'll see lawsuits over responses that AI provides. We've already seen examples of like lawyers being sanctioned for misusing results. But I think in some cases we'll see the user sue and say, hey, this, I got this response from this product and it poisoned me. Or something like that. And the companies will try to defend themselves in various ways. But I'm curious how legal departments, but also product managers are thinking about this problem in the future.

That's a really interesting question. I like, I mean, I'm sure so that the lawyer thing seems bad. Get it? I get why you'd get sued in that. Because like your job is to not do that. And like, I imagine as a doctor, like you would get, you know, you're like, I stuck my patients stuff into chat GPT and just gave them whatever pills it said. I'm like, yeah, okay. But I like do wonder if there.

Are.

So, so take, take like the, the like gen AI BI products like a thought spot or, or thought spot thoughts about. Kind of gives you a little bit of how it did it. So one of the things that Botspot does is be like, okay, we basically wrote a SQL query for you essentially, but we'll give you kind of like the, the, the framework that we use to do it so you can sort of check it. There's good and bad things about that. The good thing about that is you can check it. The thing that's tough is you can't ask sort of totally wild questions with it because it has to sort of fit into essentially the model that, that it provides. But like, most of the people are trying to use it to answer relatively straightforward business questions anyway. But there are, there are tools now that are like basically ask us anything and we'll write whatever SQL query you want. And the reason I bring that up is like, it's a little bit. The point of that is to be like, don't even look under the hood. Like, the point is like, ask whatever you want. We'll just give you an answer and we'll put a little caveat at the bottom being like, results may be wrong, but kind of we're not even going to give you the ability to look under the hood in those cases. Like, what would happen if you did that? You got the answer. You like got yourself fired. You did something. You did. Like, you accidentally exposed data that caused like a data leak. Like, you know, give me all the email addresses of people from this client so that we could send it off to for a CSM. And you accidentally sent 5,000 email addresses of like people in the government instead. Now you're getting sued. And like, that's gonna happen. Can that person turn around and like, sue the SQL bot that did that? Like, I don't know. That's actually, I would assume that there's all sorts of crazy stuff in people's terms of service saying, no, you can't do that. But, like, terms of service don't really matter. At the end of the day, it's like what the court says. So, yeah, that's kind of an interesting thing is like to, you know, people sue Tesla, they sue Tesla when Tesla crashes their cars, they say, keep your hand on the wheel. But like, it doesn't seem to work in the courts. So I could imagine you getting yourselves into some trouble here.

Well, it's funny too, because if you think about consumer products, you've got on the one hand, if I buy a pizza, it's assumed that pizza is safe, right? The food ingredients should be safe. Maybe I could choke on it. But like, that pizza shouldn't really be able to harm me. Maybe I eat too much. That's harmful. But I'm not going to sue the pizza. It happens if I buy a table saw or say Joe buys a pellet gun for his kids. Those are dangerous products. Like, you can really hurt yourself with those products. And they're in the legalese and just in our general social assumptions. And in that domain of law, the assumption is, okay, the consumer buys this, they're responsible to read the instructions and to train themselves to use this product safely. And so I wonder if at some point we'll see a similar thing with somewhat dangerous applications of LLMs where it's like, you now need to make sure that you understand the risks of using this product because it is like a table saw where it can harm you.

Well, as somebody owns a table saw, a chainsaw and a pellet gun for my kids, I can tell you as a parent, I accept responsibility for this. Right? Unless the product somehow fails in very strange ways, like the engine explodes in the chainsaw or something like that. I mean, if you cut your leg using it wrong, it's kind of your dumb fault, not theirs. So.

But that, that is probably what complicated using it in some way. It's. This is like the case that I think it is used as an example, like overactive. But I don't think in this case it was actually like some stuff went wrong, but like McDonald's thing where someone burned and got a bunch because they like got coffee still on them. Like in some. You could look at that. That's ridiculous. Like you have a hot coffee, hot coffee down to your shirt, you burned yourself like. Yeah, but I think in that case, like they. It was something, it was like it was extraordinarily hot in a way that was, I don't know, like again, held up as, like it seems ridiculous that you would get money for burning yourself without coffee. But I think in practice it was actually something that really did go wrong there.

Yeah.

And like that to me is why this gets messy is it's like, yeah, obviously if I, you know, open up chat GPT and say something and it gives me an answer, it doesn't make sense and I go off and make terrible decisions off. Like give me directions to drive somewhere and it tells me how to not to do it and I get myself in trouble. Fine, it doesn't count. But like I would imagine there are places where you like, you would reasonably think the product should work this way and it doesn't and you find yourself doing something bad.

Like exploding hoverboards.

Yeah, exactly.

Yeah, exactly, yeah, yeah. Version of that.

Yeah.

Like it was interesting. I was actually hanging out with a friend of mine yesterday. He does, he's in charge of compliance for a very large E commerce company. I won't name names. Compliance for all products. Right. So products that sell on this, on this E commerce platform, they have to, you know, the ingredients have to be approved. Right. If any ingredient is not approved, like that's an automatic red flag and it doesn't get accepted. But this person's using in a lot of cases, machine learning models to synthesize a lot of the data to comb through compliance requirements and stuff for every product there. And that's a huge task for it. But if one thing goes wrong, that's a big lawsuit because you're selling products that, you know, might injure people or cause harm. So I mean, so it's an example using AI to sell physical products or ML physical products. But it, there's a lot of parallels and in fact, who was it? We had a Kai Zenner on a couple months ago from the eu. He's the one who spearheaded the EU AI Act. And you know, that piece of speculation, I mean it's at least it was agreed upon. But there's a lot of stuff there, right, that I think will actually impact American companies and global companies as well, where, you know, models, ethics, safety, all this stuff is taken into consideration, I think. What was it? I remember seeing a blurb this morning where even the US I Think they're going to make or you know, recommend to companies that they register advanced AI models with the US government or the commerce. I thought it was pretty interesting. So don't know where this goes.

I think one real curious thing too is the international aspects of this. So the EU can try to regulate, the US can try to regulate and there's been a lot of talk about the death of globalization. But what's still as far as I know the most popular app on iPhone and Android Right now, TikTok, it's a Chinese app. And so it's how do you regulate like apps that are developed overseas, managed overseas and the same thing is going to apply to AI models. Do you try to ban them for violating policies?

The state of Montana did.

Y. Well that's true.

Yep. Yeah.

And I mean I like I, I to me that is the, the, the power like if, if they become infrastructure, like if, if LLMs become AWS is you know, which is kind of the bet I think that a lot of people are based like why is OpenAI worth a hundred billion dollars? Probably because they think it becomes sort of a backbone of technology in some way. I think it almost has to be where you don't use international versions of it. Like it, it seems we, we lose our minds over things like TikTok and, and you know, manipulation of social media and stuff like that. I think there obviously was a lot of anxiety about that in 2016, 2020. I'm sure there will be this year about basically like human manipulation of that of like we're going to create bots and we're going to say a bunch of crazy stuff and people are all gonna get riled up about crazy stuff. Okay, we're worried about that. I think Tick Tock is a much worse version of that. I don't not sort of a huge general believer in like Tick Tock is like some Chinese psyop. I think it's more of like it, it does a bunch of crazy stuff to people because it like rabbit holes them down things that are just like crazy beliefs because the way the algorithm works, but that's a different story. But the point there is like we get very worried about basically like manipulation of like how people think about these sorts of things. If LLMs are sort of infrastructure like the amount of power that somebody has to tweak things, tweak that model in completely unknown ways is crazy of like you could just sort of turn the default temperature knob a little bit and like if every text message is sort of crafted with the Help of AI Suddenly the entire company's text messages are slightly angrier. Like, and that's, that's not that far off from how that might actually work. And like that sort of thing to me is. Is like a degree of. Yeah, like there is like regulatory problems there. Like crazy. I would think that if, if Apple said, actually, you know what? Imessages all have like a autocomplete function sort of thing where we'll write stuff. Guess what? We're using like a. A Chinese LLM provide. No way, like no way that would be allowed. And, and partly because like people are just paranoid about that, but also because it's like, it does seem crazy to think that if we're worried about the potential manipulation of social media, which is a relatively manual process. Like it takes a lot of just like hammering it with bots and stuff. This is like one dial is one dial to change a model, to like change how tons of different pieces of technology sort of like think. And I think that that to me is. Is yeah like an almost. And it's hard to imagine being okay with the entire company getting addicted to a foreign LLM basically.

Lots of comments here. Too many. I don't know. Do you see anything interesting? Matt?

Sorry, I. I'm not. Let me switch.

Oh, you're not. You're not saying. To find out. I'll go find one.

I'll ask the comments.

There's one comment here that creating nationalized versions of anything is opening the door for massive bias. I would say that that's. I actually think that this is how it's going to go anyway. I spent a fair time in China, for example. Right. Like quite a bit of time there. Their idea of ethics and how they use AI is much different than the US I don't expect that they would ever adopt the US model, nor should we adopt theirs. I don't know. These are two different cultures and two different approaches to society. I don't think this is a. The approach I would take, but whatever. Quick question. How do you. How do you buy ads in an LLM product? I always imagine myself meeting friends at McDonald's just to find that a human didn't plan the event. I don't know. For the last part, I would say if you've ever used an executive getting booked by an executive assistant, this already happens. So yeah, the, the ads.

I mean, this is. I. There's probably a ton of money to be made. I remember somebody starting like a bunch of people got excited about like creating ads and Web three stuff like we're gonna build the new.

Oh, yeah, yeah.

I don't know about that, man. Best of luck. But like, ads and LLMs, there's surely there is some, you know, YC company out there that's like. We're figuring out how to inject, subtly inject ads into your LLM so that when you ask for responses, you get. I don't know either. Either. You get like a thing that's like an ad block that's like, prior to reading your answer, about like, you know, who scored the most points in the first half of a Sixers game ever. We're gonna just sell you some random thing. But you could also imagine that being like, very much product placementy, which would be kind of funny, I guess, as like, you know, we're gonna give you an answer, but there's gonna be some kind of like, overt product placement in your LLM answer. I don't know. I sort of like this. I sort of like this idea of imagining what ads look like. We're going to glue ads on it. We glue on it ads on everything. Like, there's no way. We don't glue ads on this.

Oh, there's no way. Well, the generative AI is an interesting one too, because I wonder if the AI itself is going to generate the response in the ad with images, you know. Well, AI, I love generative AI images. Right now I think we're at the sort of weird phase where it's going to go. We're going to look back on this point in history as sort of this is a. A meme, like a type of an image. And a lot of them are just hilarious. The outputs, I'm like, these make no sense. Like, faces are melting off and stuff. And it's just like. It's awesome. It won't always be this sophisticated, but it's really crude and I think it's hilarious.

So there's two. Two under later, like two very divergent thoughts on that. Is one, I saw somebody have something where it's basically like, it was like an additional finger. It was like this thing, rubber finger that you could like wear on your other hand. That was hilarious. If you want to rob stuff, like, just wear this finger so that, like the image is going to be you with six fingers. And people are like, oh, it was a JI image. Which I actually think, like, there is a pro. Everybody talks about, oh, my God. We're not going to be like, we have fake images. We're not going to know what's real or not. Everything's Gonna be fake. I actually think that's in some ways the bigger problem, that if you actually have a real image, you can just be like, wasn't me. Like, I didn't say that. Like the whole Trump, like, you know, Access Hollywood tape and stuff like that, like male. You think he would say that was him? No, he'd be like, it made up.

Oh yeah.

So like, I think like one. It's just like we start to. It's not so much everything's fake. It's like things that are real maybe fake. I don't know. The other thing about ads though is. And I, I'm sure this, this is gonna. I'm surprised this does not exist. I'm sure it'll exist soon is like ads getting. It's got to be the place where ads get injected in. Like TV shows that are personalized to you. Like in the show. So say you watch like James Bond. Like James Bond wears, I think like an Omega watch.

I don't really know watches.

Yeah, but it's like maybe you were just shopping for a Rolex and your version of James Bond. James Bond is wearing a Rolex. And like it's not like, it's like we obviously get personalized ads sort of like on the Internet or whatever, but it would be like basically the TV show evolves to be like, oh, the car that James Bond drives. It's the cars that you were just looking at. You were just shopping for a new Accord. Guess what? This James Bond drives an Accord. Like something like that's gotta happen, right?

He's thrifty. He goes back to his double wife. It's great.

Civic S.

You know, that might be aspirational for some people. So but switching gears a bit, you got a newsletter. You talk a lot about data stuff which you unpack. The tone of your newsletter has been changing lately. Used to be all about kind of modern data stack and tools and kind of more technical discussions, more tooling based discussions. And now it's, you know, Ben's take on all things business and, and, and stuff. What, what happened, Ben?

I got put out to pasture. I think there's things. One is the thing. Bit of a joke, like as. Sort of quote unquote, practitioner.

Sort of.

The in the weeds technical stuff is, you know, I'm. I'm stuck in 2019 and so at some point, you know, you're talking about things that don't matter anymore. I think more on like the tooling stuff and sort of like the data ecosystem versus kind of the broader tech world, whatever. I don't know, the, the data ecosystem was the fun and exciting thing for a bit. It was where a lot of things were happening. There's a lot of stuff changing. There's a lot of like, dynamism, for lack of a better word, in ways good and bad. You know, I think like that's not necessarily say it's all amazing stuff. It's like. But it's interesting. I think it's interesting to see this stuff evolve and like what people are trying to create when a bunch of people are trying to do it all at once. And, and you get to me in those cases. You like, you speed run through a lot of interesting ideas and see a lot of interesting things happen. And so I think like, that's an interesting thing to look at. It's, it's like you can just kind of watch it and, and funny stuff will happen and interesting things will happen and unexpected stuff will happen. And there's just like a lot of, a lot of drama in it, for lack of a better term, when the attention moves away to other things. And I think like, Gen AI has sucked a lot of air out of the modern data stack room. Interest rates have chucked a lot of air out of the modern data stack room. It's just not the IT thing anymore. There's stuff that's happening and sure, yeah, you pay attention to it and all that, but like, there isn't the same kind of like frenzied excitement there. And to me that's, I'm. I am like fundamentally not a person who is in this world because I am just like deeply fascinated by ideal data modeling techniques. I think that's kind of an interesting thing to think about as it fits into the sort of social dynamic of how companies work and what do we do about it and all that kind of stuff. But I, like, that's the reason it's interesting. Not because I like really want to know this sort of, you know, deep technical, scientific truth. And so I think like, now that things have moved away from that, it's like the more interesting kind of socioeconomic dynamics of tech and stuff are in a different world. And I think that's the stuff that's, that's kind of fun to pay attention to at least is like entertainment and like, I don't know, half this blog is basically me trying to entertain myself. I wouldn't say read it because you should learn anything. You're not going to like, learn anything practical. It's mostly like, here's some crazy stuff that's happening in the news. This is kind of fun. Let's talk about it.

Well, I mean, hopefully this is kind of the maturation cycle of the modern data stack where it's like, all right, you can't just splash out and have 2 million dollar parties anymore. It's not about just the sizzle at this point. It's about going out and selling to customers and making sure they're happy. And to your point about data modeling, maybe that seems a bit more banal, but I think the reason Joe is working on it is that's really what's needed now. And it's like, all right, we're not worried about the latest query engine anymore. People have their query engines and they just need to get things done and they need to build quality systems so their data is in good shape. So how do you model that?

Well, in fact, that discussion becomes, I think easier nowadays because people aren't so enamored with the tooling. Right. So I think it's the practice of it never goes away. But it's sort of, I mean, I feel like more like Dr. Peter Atia or somebody like trying to convince you like, you know, you should pay attention to your health, you know, but it comes a lot easier and everyone aren't used having health issues or, or you know, focusing on it versus I think, you know, whatever, I should get the new peloton bike or, or you know, go get drunk with my friends. I mean the whole thing is like the attention's been elsewhere right now from practices and I think the business stuff as well, like those discussions weren't happening during the peak MDS era because it was like why, that's boring as hell. Why talk about that? Why not talk about the battles between the big platforms when I talk about this, that or the other thing. Right.

So yeah, and that, and that's like I, to. To be like I. Data modeling very important. And, and the value nail comes probably nail that the people like me have like gotten distracted by some other thing. Like my contributions to these conversations were not useful in like a business value sense. And so I think that that to me is, is like that's the dynamic is. Yeah, absolutely. It's not saying it's not useful and not saying like the people who actually are interested in that for the sake of solving the problem. Like that's how people actually solve the problem. I'm not the one solving the problem. I'm the one who's like, I'm the bubble that people like me show up because there's a bubble. I'M not here and being like, oh, these people got to get out of the room because there's a bubble. It's like, no, I'm, I'm chasing the bubble because the bubble's exciting. I gotta find something to talk about. And bubbles are more exciting than like, you know, the, the steady slog uphill to do something useful. You know, I think there's, there's like the, the sort of Silicon Valley anti bubble types where people would be like, you know, let's just get back to work, nose to the grindstone type of thing. Which yes, that is very much true. And I think in sort of the different part of my professional life, that's kind of what Mode was about. But like this blog is more about not knows the grindstone. This is, this is go where the party is. Where's the $2 million party? That's the fun thing.

Yeah.

Which is to say like that is a distraction like that. I think a bunch of blogs like that and, and really, you know, money, companies, various things that like attract the people who were there because it's hypey is, is the thing that distracts from this actually being useful. And so, you know.

But you've always been good at calling things out, I think. And that's one reason I've always read your podcast post. I read them every week and have for a few years now. Thanks for taking up my time. But it's good. It's time. Well said. No, I mean it's, it's something I look forward to every Friday is you, you always have a, I think a really good viewpoint. I'm actually reading a post, I'm going to show a post from your blog last year around this time, February 3rd, 2023. And it was the Insights industrial complex. Right. I think that's, that's a, that's a good take. I mean it still holds true.

You.

Know, so I mean, I don't think, I don't think this has gone away since when he wrote it last year. So.

Yeah, no, I mean I, I, like, I don't, I, I, the things are, I believe it. Like, I think these things are true. And yeah, I think it's, it's not to say like my hope at least in the stuff that I talk about is like, I do at least want to avoid the, like let's talk about eight things that you need to know that happened in AI last week. Like right, there's that version of it that is, is literally just like hype chasing and I don't know that's not interesting. I think it's more that when the world is like, moving quickly, lots of interesting things get unearthed. Like, so two examples. One is I worked in D.C. for a little bit from 2009 to 2012 in like, international economic policy. And it was right after the financial crisis and that was when, like, the whole world was falling apart. A very interesting time to do that because, like, in those moments you see a lot of things that aren't like what you would see in sort of normal times. And so Matt Levine, who's like a financial columnist for Bloomberg, wrote a lot about crypto the last few years and he said one of the reasons he did it was it was basically a bunch of people reinventing traditional finance for like three or four years. And like, they, like, from first principles reinvented banks. And like, that's kind of an interesting thing to do where it's like, how's that going to go? And we decide, oh, actually we're going to reason our way into banks being a thing with a ton of money and a ton of like, craziness in the system. And I think a lot of interesting things happen when you do that. And to me, like, the modern data stack hype was a lot of that, which is very interesting. It doesn't mean there's not like, real stuff to learn there. I think a lot of ways that's the point. There is very real stuff to learn, but it's, it's sort of more interesting and kind of more fun to watch than, than. Yeah, like, I'm sure people who work in networking and it are reinventing lots of things right now, but they're probably reinventing it in like, a relatively slow and, and kind of steady pace. And like, yeah, that's interesting if you're really deep into the networking world, but it's not interesting if you're interested in the gossip and drama, which I think for the blog anyway, is half my appeal, half my job.

Let me ask you a question that's related to all the current hype, but I think has bigger human implications. So hopefully it's a question that will be interesting even five to 10 years from now. And that is back in the 90s already we had things like grammar checkers, spell checkers, and back then there were these studies that showed that that changed how people wrote and there was a lot of panic over that. Like, all of a sudden, you know, grammar becomes more uniform, maybe a bit less interesting. People don't use interesting syntax as much as they used to. And so you as Joe said are a good writer. What's your opinion on the impact of Genai on writing in the future? How is it going to change this human activity of writing down the road?

So I recently saw like a podcast with David Perel who is a self described writing guy on the Internet. I have opinions about his brand but we don't need to get into that now. And, and there was a thing in there to me that was somewhat alarming that like about this alarming. This is a little very much like an old man take. But there a lot of people say like writing is thinking and I think that's generally true that like you have an idea, it's usually a roughly formed idea, like I'm going to write something about it. You write it down. You realize the idea sucks. It takes forever to figure out what the idea is. And like the writing process is, is the process by which you actually kind of realize stuff. I think that's generally true. One of the things that like, like the say Chat GPT today does is it lets you basically be like I'm going to give you a handful of bullets and you write the rest of the thing. And you can essentially shortcut that. Like you can shortcut the, the thinking part because it'll fill in all the details for you. It doesn't. It does fine. It's like whatever. It writes a good high school paper or whatever. To me it's not so much like that. That sans the language down into something generic and vanilla it kind of does. But like okay, whatever. Honestly, most people in most contexts when they write like there aren't that many people who are writing anything that's not that anyway, like because you're writing email like most people aren't writing much anyway like writing emails or writing messages and work. It's like all kind of fine to be generic. Like as long as it's, it's telling you what you need to know. Like that's fine. Most emails I write are pretty boring, but writing it does force you to think about it. And I think if you just can be like I'm going to give you kind of these bullets, then you never actually like think about what you're saying. Like, like Jeff Bezos has this whole bit where he like makes people write memos and the memos have to be not bullets, they have to be pros. And I think that makes a lot of sense because like if when you write pros you suddenly realize when these ideas don't connect, when you write bullets, you can skip everything. You basically like three points. I think they're good enough for the bullets in this little bullet. That's enough and it works. I don't think that like that doesn't really constitute an idea. It doesn't constitute like a, like a current thought kind of just like a few dumped out of your head and like hope to connect and that GPT connect them for you if they don't really make sense. So I think it ends up sort of short a lot of things there and it also be a little horror. They're like point of horoscopes. You read them and generic enough can see everything in them that is like oh my God, predicting the future set of stuff like you're gonna thing that happened to you that means a lot your past. And like okay, you will suddenly see that in your day. And I think there's some of that too where like you can ask it questions and stuff and it'll give you these sort of generic ish answers that you start to be like oh my God, this thing saw this brilliant stuff when it was really just kind of like a, you know, it was giving you kind of a bunch of proverbs that you can always kind of see yourself in proverbs, you can always listen to pop songs and you're like oh my God, this is about me. I think it's sort of that. And so I think there will be less of like we become boring writers and more of like we become short circuited thinkers.

It's a key point. I mean give you an example. Today I was, you know, I was writing a part of my book as I do every morning and you know, definitely hit a stumbling block with connecting a few ideas and, and that took it's quite a good hour just ruminating on it, right? Getting a legal pad and a pen and just sketching out the ideas and the flow. And that's. I think you're absolutely right. The, the ability to connect different ideas together to form, you know, original thought is. I think it's it. I hope it doesn't go away. I mean it will change for the vast majority of people and I guess it depends on what you're trying to do. Maybe, maybe for lower, low impact, low quality thoughts you need to do like just a quick email response like thank you. Sure. That could be auto responded. I don't give a. But like yeah, when you're writing a blog post for example, I mean you can already see it and some people, some people will make LinkedIn posts about. Here's one, here's why you need to use Apache Spark. Here's five reasons. And you can tell it was generated by chat GPT. There was nothing interesting in there at all. You know, same with responses I see, you know, even comments or on social media, whatever. So that's, that just shows to me that you're lazy as hell and you're, you know, not putting the effort. So I mean, on one hand it also means I hate to use competition in the context of content because I don't really think content is competitive. But it with the perspective of with respect to time though, it is competitive. Like we're competing for attention and so high quality thoughts. Like the stuff that all of us, you know, generate, I think that there will be sort of less competition but at the same time discovery and curation of those is going to be harder because there's just going to be a sea of just crap out there. It's interesting.

Yeah, maybe this is kind of the future of really high quality search. So for example, a lot of people have been commenting about how Google search has degraded and so specific example, I was looking for the Day to day Texas page two days ago to find the agenda typed in Day to day Texas. I got like 10 results above day to day Texas that were not Day to day Texas. Same thing on Amazon. Like you do a search and most of the stuff you find is garbage. Even if you spend search for a specific brand, it will be way, way down. And so the question is, is there going to be a new generation of search surfaces actually high quality human content, or is it more about curation? Is it more about the newsletters where someone has actually gone through and found the best content that's not simply LLM generated?

Yeah, I mean I've been using Perplexity a lot lately. I think it's a great app. So. But yeah, I don't know. What do you think?

Well, I mean I think that does. I know some people who, who have basically started just searching exclusively. Like their question Reddit.

Interesting.

Where essentially they're just trying to search Reddit for stuff because it's like if they want to know, you know, what's the best, I don't know, I'm trying to buy a new microphone for a computer. What's the best one? Like I, if I search best microphone, I'm gonna get a whole bunch of like content aggregators or sites that are trying to sell you stuff or what. Like now AI generated sites that are like here's the best microphone. You know, it's not useful but if you search Reddit. Reddit presumably hasn't really been invested by like bots yet. Like the, I don't know why you would do that on Reddit. Like the. Seems like people, I don't really use Reddit, but it seems like it's mostly like, you know, why would we hang out here and then have all of our conversations be like have chat GPT write my answer. Like that's not the point. Like it's a social thing and so we're actually have real conversations and so it's like much more real. And so I could see there being something, I don't know, like it's not necessarily Reddit search, but something that tries to be a way of cleaning up like the AI generated noise in the same way that Google tries to, you know, like down rank what they perceive to be basically like content aggregators or whatever else, you know, like how do you find high quality content? I don't know. I don't know how you actually identify that. I know there are people who are, you know, various companies have raised tons of money saying they can do that, but I don't know if that's, that's true or not.

Yeah. And it kind of reminds me of the notion of food deserts in, in urban areas. I think this is what's going to happen with content where the vast majority of people who can't afford to get curated content, if that's, if you're paying for it, for example, you're just going to go to whatever crappy. You know, it's no different than going to your bodega and living off of Twinkies at that point. It's, it's the same sort of substance or lack thereof. Whereas people with the means to get high quality information, you know, I guess it's similar to Whole Foods or something, then you can do that.

But so that's kind of interesting. So it's like a, a, the Internet is like democratizing access to information. But then somehow this like suddenly everybody comes to New York Times, they need to have paywalls because they're worried about things like getting scooped up by LLMs. And so now we have like a undemocratization of information where there's just like low quality stuff that's free and high quality stuff everybody has paid for.

I believe that's the one scenario. I mean I, I calculated subscriptions, I pay for various news outlets and I think it's around, I think 900 a year. But a lot of people can't afford that. Right. So that's A lot of money to most people. So. But you know, except like the information. Right. That's expensive. That's an expensive news outlet. Certy I pay for that. That's just a newsletter. That's 120 bucks a year. I value Ben Thompson's insights enough, I'll pay for it.

Right.

And there's, there's many other news outlets but it ends up costing a lot of money. So.

Yeah.

Yeah.

I thought about that as like the, the sub stack model. Like this is kind of. The sub stack model generally is like we want to make people who write stuff the Ben Thompson's the world be able to. To make money off of it. But I guess there is a downside of that which is if everybody does that then it becomes somewhat unaffordable. Like unaffordable for, you know, you don't have access to that much.

Let me ask you this. Why don't you charge for your substack?

People would expect it to be good.

That's fair. I don't either for that reason.

Yeah. You know, you get what you pay for. And this way I can tell people that. I mean I like partly because like I don't. I don't know. I don't. It's the, the money wouldn't be that much much and you know, I thought spot graciously lets me write it and so I'm able to mode and mode and thoughts about you know, I, I have a day job basically and so I don't need it for like it's not my, my living. And I kind of view it as like the, the interesting thing to me about it is like I'd rather just people be able to read it and have conversations about it than like how do I, you know, cut its distribution way down to make a little bit of money. It's like it's not worth it to me.

Yeah. Somebody wrote here too. There's also a real problem of inclusion here. Having the privilege to have the time to write a substack is something a lot of folks don't have. Yeah. Diversity of voices. I think it's going to be a real thing. Right. And it's so it's an interesting one. I, I don't want to name drop, but chatting with a Colin Kaepernick a couple weeks ago and he's working on some interesting stuff. Giving underrepresented voices a chance to be heard using generative AI with a new company he's working on. I think that's a cool approach. And he started this whole Thing because he had a book publishing company that was just gate kept by the big publishers. You know, I mean, he's, you know, if you know who he is, he's a fairly. I wouldn't say he, he's controversial in some ways and that, you know, he's a civil rights activist and I just support what he does. But you know, a lot of the big publishers are like, yeah, your stuff is we can't, we can't publish this. And so he started his own publishing company as a result. Right. And so he, but he wants to give underrepresented people a chance for them to, you know, create cartoons and comics and all this stuff on this new platform. I think that's a cool thing to do where you're using, you're leveraging generative AI to hopefully give voice to people and distribution to what they want to do. I guess YouTube is sort of like that too. But yeah, you got to so create your own content, which is a big barrier for most people. So.

Yeah, and I, and I actually think like, like on that point in this sort of the points about like, how do you get more people to create stuff? I actually think it's mostly a distribution problem, not a creation problem. Yeah, that, that creating stuff can take a long time. It can be relatively. It's, you know, different people have different, different versions of that. But like the hardest thing is, is how do you get, you know, anybody who writes on like the New York Times opinion page is going to have incredible distribution that a million people see everything that they write. And so I think it's more of things like the Colin Kaepernick thing to me seems like, hey, if we have a distribution platform, a publishing platform, whatever, that, that gets people's stuff in front of other people. And like in some ways things like Twitter or substack or social media or whatever, it's all like semi meritocratic in the sense that like anything can go viral or whatever. But in practice it's not really because they're these kind of networks of people that will promote stuff. Not, not in a sense where like they're doing it necessarily by design, but just like there are elite circles in Twitter that everybody talks to each other. And so like, how do you break into that? And I do think, like ways to help more underrepresented groups get into those distribution circles is actually more important than helping people create. People will create. If they get distribution, they will create. People are very, very capable of that. I think the hard part is, is the sort of social structures that prevent distribution.

Absolutely. Well, cool. What are you excited about for this year then?

What? Anything.

Got anything else going on?

I mean, I think. I don't know. I think that. I think, like, whatever happens with the Generative Eyes stuff will be interesting partly again, because it'll be like another frenzied spectacle of, Of a bunch of people figuring stuff out pretty quickly. Better or worse. I think that's like. That makes for interesting stuff. You know, I would say I am not excited about, say, like, the 2024 election and. But that's, you know, more terrified of. Of like just the. That plus all this AI stuff to be seems like a recipe for a real mess. And so, you know, hopefully we figure out some things to do there that make this not a disaster by the end of the year. But yeah, you know, I think like, just the, The. The ability of, of these tools and technologies and stuff to create such enormous confusion to me is like, really? Let's, let's see how this one goes.

So we'll see.

I don't know.

Totally agree. Should be fun.

I. I hope we're going into it at least a bit more cautiously than we went into, say, 2016, where everyone was very complacent, like, oh, you know, don't. Don't worry about it. It's fine. Nothing bad, bad will happen. We'll see. We'll see how it goes this year.

No, no. Yes. Oh, what do you say, man?

Even, like, that's like. Yeah. I think there's like, the particulars of like, you know, Trump that people probably are more clear for. But, but I do think, like, the quickly sort of the, the landscape on which this happened and like, basically political work are very much still to some extent tied. Like, politics happens as media and the nightly news, like New York news, the big thing to pay attention to and stuff. And it's like, like Tick Tock. Like the degree to which Tick Tock is going to influence this and not saying, like, influence in terms of manipulation, but just like, like Gen Z gets their news from TikTok.

Yeah.

And what do they read on TikTok? Who knows? Because everybody's TikTok feed is entirely different. Like, I have no idea. Like, there isn't. You can't even look at, like, the way that, oh, look at our Twitter feeds are all biased or whatever. My TikTok feed is completely different than your TikTok feed, which is completely different than Joe's TikTok. It's like, stuff like that where it's like, how does that. I have no idea how that changes, but it just makes it, like, way crazier. And so I think, like, a lot of. A lot of weird stuff could happen because of that. And it's not. Not related to Trump or Biden or any of that. It's just like these things now happen in ways that are kind of unfathomable eight years ago or 12 or even.

Yeah. Like the Gen Xers amongst us can't even process this type of media.

Right.

We had our own new media that was in the 90s, and now it's just a whole new generation that none of us can even comprehend.

I mean, there's a whole book written about it with Douglas Copeland. Gen X. Right. I mean, we're. We're dinosaurs now. So.

Yeah. And then the millennials, and then.

Yeah, yeah.

All was changing and we can't keep up.

I'm not sure, you know.

Cool.

Ben, it's been great chatting with you. For people who want to learn more about you and what you're up to, how can they do that?

There's the blog, basically, is the best things. Ben.substack.com there's, you know, LinkedIn and Twitter for. If you want me to periodically post links to a blog, which is basically all there is, or just email me. I don't know, there's emails on those various websites if you want to just, like, talk or hang out or whatever. I do that sometimes, too, so that's awesome.

Well, cool, man. Good chatting with you.

You too.

Actually, see you next week.

Yeah, I'll see you here.

Okay. Awesome.

All right, have a good one.

See you. Bye.

Bye.

Take care.