Why Google Search Isn’t Going Anywhere Anytime Soon | Bilawal Sidhu

I'm still very bullish on Google because I think it's like the tip of the iceberg of what you see in tech companies. And the submerged part is just amazing. They've got the most complete models of the physical and digital world. They've got ubiquitous distribution and an existing ecosystem of ads and sales to plug monetization in. So I think it's still a magical combination.

Hey, welcome to the Next Wave podcast. I. I'm Matt Wolf. I'm here with my co host, Nathan Lands. And with this podcast, it is our goal to keep you looped in with all of the latest AI news, the latest AI tools, and just help you keep your finger on the pulse so that you are prepared for the next wave of AI that's coming. And today we have an amazing guest on the show. We have Billeval Sidhu on the show. He is the host of the TED AI podcast. He's an ex Googler. We're going to talk to him about what it was like working on AI and visual effects over at Google. We're going to talk about the difference between whether or not we should be accelerating AI or slowing down AI. We're also going to learn about how some of these AI visual effects tools work, because this is the field that Bill of all worked in for so long. It's an amazing episode. You're going to learn a ton and can't wait to share it with you. So let's jump on in with Bill of all Sedu. Thanks so much for being on today.

Bill of all, thanks for having me. Gentlemen, pleasure to be here.

Yeah, so I want to just kind of dive right into it. Your background is Google, right? So I think when you and I first connected and we first started having some chats over Twitter dm, you were still actually working over at Google at the time, and you also were kind of doing a creator business on the side with your YouTube channel and everything you had going on. But Billiston, what were you doing over at Google? What was your role there? You know, what was your experience like over there?

Gosh, yeah, it was awesome. So I've spent a decade in tech, six years at Google. I've been able to work on projects that blend the physical and digital world. And I started off in the AR VR team, really, when Spatial computing, as it's now called, was first popping off. This is like right after the DK2 came out. Google Glass was a thing and everyone was talking about what is the next iteration of computing platforms? Where are we going to go from like this mobile revolution. And so I had a chance to work on a bunch of cool stuff there. YouTube, VR content, live streaming, Coachella, Teen Choice Awards, Elton John, the camera systems that we used to do, stereoscopic 3D, capture, AR SDKs. When that became popular, augmented reality sort of hit the scene. And then after that I spent four years at Google Maps, basically creating a ground up 3D model of the world, sort of remapping the world if you will, and then turning the world into an AR canvas with the ARCore Geospatial API. It's been a lot of fun and yeah, it's been awesome to work with some really talented folks to work on these projects that have been blurring this line between the world of bits and atoms.

So I'm curious what like working on all this stuff in my mind, I can't even like imagine what like a day to day looks like at Google. I've been on the Google campus and you know, it looks like a giant playground for tech nerds. So I'm, I'm just kind of fascinated by like what it's like to work at Google. Like what does a day to day look like over there?

You know, So I was a product manager and so day to day for me is going to be very different than if you go talk to like an engineer or designer. For me, really it was a lot of meetings, let me be perfectly honest. Just like a ton of ton. But there's some very cool things I think like Google and big tech companies generally are sort of this like interesting microcosm where, where it's like, you know, I'll send out an email and the, like the guy that wrote the book on computer vision, like the computer vision book that like everyone reads like responds to it and I get a bunch of pings being like, oh, so and so responded to your thing and it's like all these Pokemon that like, you know, these companies have caught that are available at your beck, beck and call to share ideas with, pull into your own projects and really just like, you know, there's such a, it's like the tip of the iceberg of what you see in tech companies. And the submerged part is just amazing. So it's like when I moved over to the Maps team, I was thinking of working on glasses at the time. And the reason I went to Maps is like I met this engineer who's like, oh yeah, we write Cls to move satellites around in the sky. And I was like, wait, what? Huh? You move satellites around the sky and it's like, yeah, like literally orchestrating a fleet. Like, you know, like most people don't know this. Like, Google owns their own fleet of like, not just street view cars, but airplanes. Oh, wow. And so like the AB to like task those to like, hey, we need to, you know, we've got this like Sundar IO thing coming up and we're going to be presenting immersive view. We got to go capture this high resolution model of London and like not, you know, and suddenly things in the world of atoms are moving to make that happen. I think it's like absolutely amazing. I think people deeply underestimate the data moat that Google has. Obviously the most complete digital twin of the world we're talking about. But like search right, like you YouTube. Oh my goodness. All this stuff is available. And so like, there's cool things and products you can build around it, but along with it comes, you know, which may not be a surprise to people in tech, but like a ton of responsibility and sort of guardrails for how you actually use this data. So it's like the size of the prize and the data sets you get to play with are amazing. But to be able to do stuff with it, you really, really have to be exceedingly thoughtful. And there's a lot of process involved in, in unlocking that innovation. So, yeah, that's how I would describe it, I think. Like, it's just like a. It's like, it's like a Disneyland for nerds, to be honest.

So it sounds like you're still really bullish on Google because I know we were having like some playful banter last year about like, you know, I was like, maybe Google's going to die. You're like, what the hell are you talking about?

Yeah, yeah, we've. We've actually had that conversation on this podcast before where I'm like, you know what? I think I give more credit to Google. I think Nathan's a little more. I don't know if they're going to be like the top dogs in AI. Like, where do you stand on that? Do you count Google out? Do you think Google will pass the Microsoft AI, you know, Avenger mega team?

It's hard to say anything with certainty, but what I will say is like, you know, after I left Google, I think I was like one of the few people I felt like I was in isolation saying good things about Google. Everyone was just like, oh, yeah, they're just too slow. They fumbled. They came up with a transformer. All the transformer folks are left. I think it's a situation where there was no real disruption in sight for the search base. Yes, there were talks about like, hey, like kids are like searching on TikTok and like YouTube like now, but YouTube is owned by Google. TikTok is kind of short form. Are people really going to be like, is that going to be a resilient thing? And one might argue now social networks are places where people do a lot of searching, but traditional search sort of as like, you know, you know, maps guy. To just give the maps analogy, it's like, you know, like maps is how you discover stuff in the real world and Google is how you discover stuff in the digital world. It's literally your window to the world Wide Web. Right. I don't think anything had like sort of questioned the strong position Google was in in that regard until ChatGPT came out when suddenly people could start connecting the dots. And CEO like if you connect large language models with like, you know, Knowledge Graph and search index, kind of like perplexity and you know, Microsoft Copilot and whatever the heck else OpenAI is going to announce in order to kneecap any, any Google limelight next week, like I think people start saying, hey, there's a disruption inside. And I think combined with the fact that like the search ads business model was just such a money printing machine and still is, and the fact that, you know, the cost per query of these generative AI models is obviously going to be higher and how do you do advertising and attribution and all this stuff like that, it would kind of represent a contraction in the money printing machine. And the pie and the business that Google created had all the signs of sort of innovators dilemma. And I think Google has sort of adopted the playbook of the innovator solution. And initially they had some of these reorgs that felt more like exec reorgs and now they're actually bringing together the Brain and the DeepMind teams and they're actually shipping at a really good cadence. And I think they still have some of the most unique data sets that other folks are talking about, you know, that may or may not have been scraped. You know, case in point, the CTO of OpenAI being asked by Joanna Stern about what does it exactly mean that you train on publicly available data. So all this to say I'm still very bullish on Google because I think they've got the most complete models of the physical and digital world, they've got ubiquitous distribution and they've got the right infrastructure chops to basically like bring that cost per query down and an existing ecosystem of ads and sales to plug monetization in. So I think that there's a like, monetizable sort of like answer engine model. I think Google is one of the few companies that could crack it. That isn't to say that I think like OpenAI and Microsoft can't take meaningful market share. But let's be honest, how many of us actually use Bing? Like, I don't. Right. Like, I used it for a little bit and I probably use Perplexity more now.

Yeah. I mean, I am using ChatGPT and perplexity instead of Google a lot these days.

Agree. Me too, to be honest. Yeah.

And then, you know, I've been following, you know, a long time ago, I used to do SEO like a long, long time ago and I've been kind of following that space and like, like in the last two months and I kind of predicted this a year ago. They're, they're like having major changes to the algorithm where they're really focusing on authority.

Yeah. Domain authority. Yeah, yeah.

The reason they're having to do that is because of all the flood of AI content. Right.

Good Lord.

They just can't deal with the flood of AI content. So it's like, okay, how do you deal with that? Let's go back to like really valuing the big brands and the big names or the famous. Or the famous people too. That's the other thing is maybe like they're taking social signals. Like you have a lot of followers on social media now. That's a signal that you're, you know, an author they should listen to.

Yeah. I mean, why do you think we signed on with HubSpot? We want that backlink domain authority. That's the only reason.

Yeah, yeah.

I mean this is like, this is going to be a meta problem though, for the industry. Right? The just the explosion of synthetic content. I mean, and some social networks are almost like incentivizing it. Like, like on LinkedIn, it gives you. It uses GPT4 to suggest a comments. And now you have just the most cringe like, like regurgitations and summarizations of like the original post from like, clearly a normal human being would never write this. But like if there was a meme about how to respond to somebody on LinkedIn, I mean that's like encapsulates the style. I see.

And so, but that's happening on Quora too. So like right now Quora is starting to rank at the top of Google results and Quora is being dominated with ChatGPT responses now 100%.

I mean, like books on Amazon too. And I think the way they came out with a restriction as well, you can only upload X number of books per day. Like, I don't know if that's the solution, but like, this is like the deep fake, shallow fake problem too. It's like everyone's talking about detecting deepfakes and like, how do we figure this out? It's like, well, the thing that causes maximum harm today aren't actually deep fakes. They're like super shallow fakes where you take a photo from a different time or a context and like, you know, kind of put it against another context. This is exactly the type of stuff you'll see, like community noted on Twitter. And that stuff's like, relatively easier to detect because you can actually find the source imagery if you do, like reverse image search. And so like, you add in the generative problem on top of that, it's like, even crazier. But like, most platforms haven't really even solved the shallow fake problem, right? It's really like when it reaches a certain threshold of distribution, there's sort of this retroactive, let's go throttle this thing versus, like, how do you get ahead of this? Anyway, I could talk about that forever because some of the ways to avoid that is like ubiquitous surveillance, which is also not, you know.

Oh yeah, sounds just sounds great.

It's like somehow the solutions to things that sound 1984 is like 1984 technology. It's like, it's kind of weird how that works.

But it's funny, somebody shared an article with me just like the other day, that was an article about, well, not just me, but it was like an article about like these seven AI influencers are, you know, changing how we see AI or something like that, right? And then like, one of the seven, like, was my name and I read the blurb about myself and the blurb about myself was like, I grew up in Louisiana doing real estate. You're my neighbor, somehow. And like transitioned into computer programming and then started teaching AI. And I'm like, other than the fact that they're like, matt makes content about AI, everything else about that was just completely wrong.

I think that's where also these models need to be anchored in some sort of real knowledge graph. And that's not to say that an approach like search is only going to give you the truth, right? There's what is even the truth and there's differing opinions on it. But I think these models to be able to just Kind of fact check themselves at least with known information and come up with oh, at least three sources that are reputable are saying this is Matt's bio would be better than I don't know, some like this is the equivalent of like SEO dribble that sort of started bleeding into Google around like 2019 I think. Like, you know, it's, this is going to be a huge problem and not to mention the implications of like if we've run out of content on the Internet and we are actively disincentivizing human generated content, like how are we going to train these models? Like what's going to happen there?

Yeah.

So HubSpot just put out this really cool checklist. It's called the AI Adoption checklist. And if you're trying to implement AI into your business in any way whatsoever, this checklist has you covered. From security to privacy to planning to training, this has you covered. Click the link in the description below and grab the checklist. Today I want to, I want to talk real quick about, about search because just a peek behind the curtain to anybody who might be listening to this episode. We're actually recording it right before Google IO right. Like Google I O is next week from the time we're recording this bill of all and I are actually going to be out at Google IO attending in person. But you know, one of the things that you mentioned is that OpenAI and Microsoft have a tendency that whenever Google announces something they need to jump in and like sort of one up them. So you know, by the time this episode comes out, we'll probably already know what OpenAI did with something search. But the rumor right now is that OpenAI is creating some sort of their own search engine with maybe with Microsoft involved, maybe not. There's still a lot of rumors and speculation flying around. But knowing what you know about Google, do you think OpenAI and Chat GPT can come in and compete?

I think no. Like I think the, the search index Google has is a very strong moat. The fact that they can sort of almost like map the Internet in like almost real time is like just a hard technical and infrastructure problem and they're really well set up for it. I'm curious what it is. Like obviously this is complete pontification, rumor mill. Like what it is OpenAI is going to roll out. What I think it's going to be is like something at parity with perplexity with maybe a better search index involved. And even if it's something like that, where you get sort of this multimodal summary where it looks at a bunch of links. You get some images, maybe some embedded videos, and a summary of whatever it is you asked for with citation. So you can go validate sort of the quality of those, like, the links that were summarized. I think that would be a huge step up, right? Like, just getting. Just being able to invoke like. Like search inside of ChatGPT right now is clunky. You have to be like, well, and look this up, research this, and, like, explicitly prompted to do that. And being able to do that in a fashion that is, like, really about leaning on, like, sort of real time and like, sort of, you know, content that has real provenance along with, you know, like sort of the distilled wisdom. You know, the wisdom is debatable. The distilled wisdom that is in. In these large language models. To summarize that, I think is still a magical combination. And I don't know if you all feel like this, but I think the vibes have been shifting with regard to just the conversation and sort of. Matt, I know you had this, like, post about, like, we think things are slowing down, but here's a bunch of announcement events coming up. Right? But doesn't it feel like just like those leaps that we hoped for haven't quite come? I guess Sora was kind of a leap. I think the trend from like a million to 10 million to maybe infinite context is like, an interesting leap too, but maybe we're just getting too used to it, you know, like, versus the technologies, like, the pace of advancement slowing down. Curious. What y' all think?

I don't know. I feel like it's kind of like it's on social media. The perception is, you know, the vibe has shifted. But I feel like a lot of people who actually are closer to Sam Altman, they don't feel that way. And so I. So I still believe that, you know, they have something amazing coming.

I follow. Gary Marcus. I know. I think you actually have Gary Marcus coming on the TED AI podcast, so we'll probably get some more insights from him in the near future. But, you know, the sort of Gary Marcus thing that he's been sort of all over Twitter about is that we're not seeing that same leap from GPT4 to GPT or the same leap from GPT2 that we saw the GPT3. He's sort of arguing that that exponential curve that everybody's talking about that we're on with AI is not true. Right. We're not on this exponential curve. Otherwise, like, why didn't we get from GPT4 to GPT5 and half the time we got from GPT2 to GPT3. Right? Like, why is, why is it not showing that? But I also, like, I think my counter argument to that is just because we're not seeing it doesn't mean it's not there.

I think there's a lot of stuff happening, like Nathan sort of alluding to behind the scenes over at OpenAI that we're not seeing. I think it's probably more related to like compute requirements to actually use some of these newer, more advanced models. If they were to release it right now with like the compute that's available, it would be like really expensive. And you know, the 20 bucks a month that people are paying to use ChatGPT is probably not going to cover the cost of inference to run, you know, these newer models. Same thing with sora.

I mean that's kind of the leading theory. I know that I believe with the GPT2, like the mysterious model that came out and it's like, what the hell is that? Maybe that's what it is. Maybe this is something they actually developed like a year or two ago and it's a more efficient architecture or something like this and possibly that's what GPT5 is built upon. Then they have less issues with the cost issues in theory.

But I think it'd be hilarious if the GPT2 stuff is actually OpenAI. I mean, what an interesting way to sort of test a model in the wild versus, I don't know, running some sort of AB where like. Or some sort of like experiments on the ChatGPT website where a subset of users get a certain model versus another set of users. Maybe like making it explicit that there's these two versions of the models and having people respond to it separately is interesting. Maybe there's an intention to create a bit of a pr, like sort of like, you know, kind of seed the conversation and kind of like grease the wheels before the real like sort of, I don't know, race car jump moment happens or whatever. I don't know. But it's like, yeah, like I'm inclined to agree. Like there's the compute stuff. Certainly SORA hasn't been rolled out widely because like it's just so compute intensive. Right? Like you need to come up with a completely different like pricing model way beyond the $21, maybe hence them talking to studios and stuff like that. But all these models will get optimized too, to the Gary Marcus point. It's interesting. I had a conversation with him last week. And it was like. It's like he's been very consistent about, like, this not being the right paradigm. And, you know, I think people like to. Like to. For the lack of. Just put it bluntly, people like to shit on Gary a lot. But, you know, if there's one thing Gary's been, it's like, it's been exceedingly consistent. And so, I don't know, like, I would like to see this sort of agentic co pilot that feels more like an employee. We certainly haven't seen it yet, right? The converse of everything else is the expectations around AI. Like, a year ago, if we go back to when GPT4 came out, were just so fricking high that people just thought, like, whether you were a knowledge worker or a visual creator, you would look at the narrative and you'd be like, holy crap, like, this thing's going to take my job. And then, I don't know, if you all saw the tweet, I was like, and then you go, use the tech. And it feels less like this, like, fricking Kaiju Godzilla that's going to stomp you and more like this, like, chaotic golden retriever that you can kind of coax to do cool stuff with you. And I think that that delta between expectations and reality is so stark. And, you know, like, there was all these. I think even in a recent Sam Altman interview he was asking. One of his biggest regrets is, like, GPT4 didn't have that, like, economic impact everyone thought it would. And he's worried that the pendulum will swing sort of the other way now where if expectations were so high, people were like, oh, yeah, whatever. And so, I don't know. I think the answer is always going to be in the middle. But I can't help but feel like we've gone past the, like, peak of inflated expectations and we're going into the trough of disillusionment.

Well, we'll see. I mean, like, Sam Altman also said that, like, he was surprised that GPT4 was so successful and that it, you know, it kind of sucks.

I don't know.

My. My feeling on that is that Sam Altman is one of the, like, greatest marketers of our time right now. And he's, you know, he's really, really, really good at getting that hype wheel spinning.

Oh, baby. Yeah.

I swear. I think. I think Sam Altman is just really, really smart. And I think, you know, when you see him speak, right, when he does interviews, he's very calculated, right? He'll ask a question. And he'll sit there and he'll usually pause for a good few seconds before he responds. And I think that he's got that marketer brain, like, what, what can I say is going to sort of spread the flames to hype this up a little bit more. And I think that's kind of how his brain operates. So I think, you know, him saying, like, this is going to be the dumbest model you've ever used by a lot. And we're going to, you know, we're going to look back at this and be embarrassed by what we put out. I think that's all marketing.

I'm certainly excited. But, you know, if I could have one request right now, it's like, just give me GPT4 from April 20th of last year. Give me that vintage of GPT4. It was better. It was better. Damn it.

Can't you still go into, like, the OpenAI playground and, like, select the older models?

I think you can, yeah. And I feel like just all these models, especially in the consumer interface, follow this trajectory of, like, when they launch, they're really good. And then over time as like, you know, various efforts to make sure the output is, you know, on rails and not harmful kick in. You see this deterioration happen. But, hey, that's why we also have open source. Right? Right. Yeah.

Well, let's talk about visual effects too, because that's like, that's really your background over at Google. I want to go, I want to go back, like, to like the sort of, you know, 3D imaging 101 for a second here. Can you sort of like break down the difference between things like photogrammetry, lidar nerfs, and, you know, Gaussian splats?

I would love to. In fact, the thing I was talking about is everyone talks about generative AI a lot, but I think the part that's getting not that much attention is this visual spatial AI space. And so think about spatial intelligence as really just reality capture. The world is in fricking 3D. Right. And so, Matt, you nailed it. It's like, basically photogrammetry is the art and science of taking 2D images and other sensor data like lidar, and turning it into these 3D representations of the real world. So photogrammetry has been around since before computers were invented even. This is a way of basically using math and images and observations of the world, really, to extract 3D structure from it. But you should also think of spatial intelligence as the ability for machines to interpret the spatial Data like maps, 3D models, the world as we see it, right? And so to me, like photogrammetry or reality capture, all these other techniques are all about recreating reality. And so photogrammetry isn't new as I alluded to, right? But I think what's gotten a huge boost and why you hear about all these things is like thanks to machine learning, basically like these learned approaches to modeling the complexity of reality, right? Like, basically like how do I take a bunch of 2D images of the world like this and essentially have a model do this inverse rendering problem where it's like, oh, here's where these hundred photos are located in 3D space. Based on this, I'm literally going to eyeball ray tracing and create a 3D representation that makes sense. And since you know exactly what the model looks like at the photos that you've taken, the representation that you get eventually is like good enough from all viewpoints. And so like this basically the first NERF paper dropped in 2021 called Neural Radiance Fields. And then there's just been insane progress like we talked about, from like data centers to like the GPU and your fricking like, you know, Nvidia workstation to like the iPhone in your pocket. But even this wasn't new. There was like spiritual successors to these like ML based learned representations to sort of encapsulate the complexity of reality. Enter Radiance Fields, right? Like the way like, like think about radiance feels generally sort of like imagine like a, a voxel grid, a cube of cubes where every single cube has a color value and an alpha transparency value. And that's kind of what you end up getting with a nerf. And then when you do volume rendering, you can basically end up getting these photorealistic renditions of the world. And so the cool part about Neural Radiance Field is instead of photogrammetry, where you get this 3D mesh model with surfaces with textures plastered on it, think of it like crappy GTA looking models. What you get with nerfs is like a radiance feel. This voxel grid of like all these voxels and their various values, they're in that change based on how the camera is looking at it. And because of that you get all these things that photogrammetry couldn't do, which is like modeling transparency translucency, like fricking like glass, like shiny objects. All this stuff can be done. Fricking fire. Volumetric effects, all the stuff that photogrammetry can't do. Because imagine needing to come up with like a cardboard paper Mache model of that thing, it's going to look like crap. How do you model hair, fire, fog, all these things? And you can do all of that with these implicit representations. Now the problem with NERFs were the rendering speed. Because you've got this voxel grid and you're doing this volume rendering where you're like first doing like the training process takes forever, but then when you want to render an image, you got to do volume rendering and like trace like these rays through that voxel grid and add up these like values. Like that takes a lot of time and basically it's like, think of it like one frame per second to render out some of these videos, right? Along comes Gaussian splatting, which is like, hey, do we even need the neural part of radiance fields? Like, do we need like ML? Can we just do this with like old school statistical techniques and like, which is kind of wild, right? And so instead of having this implicit black box representation where like reality is modeled in the weights of this like mlp, this like multi layer perceptron, you've got this explicit representation of these like ellipsoidal splat looking things called Gaussians. Just think of them like super stretchy, like fricking spheres. Like, turns out you can get like a huge jump in quality while also being able to render way, way faster. And so like it's like from One FPS you're getting like 100 frames per second. And since it's an explicit representation, it's in this like most formats, like all these apps that I'm showing on the screen, it's in this format called ply, the Stanford ply file. You can basically bring it into any industry standard game engine. Like you can bring it into Blender, you can bring into Unreal, into Unity. And since it's not this like black box, like this neural network that you have to deal with and it's explicit, you can go and delete and edit things far more easily. And so it's like super crazy to see like what's happened there. But basically between nerfs and Gaussian splatting, think of Gaussian splatting basically as radiance fields without that neural rendering part. And the paper used the terms like training or whatever, but there's no neural networks involved at all in 3D GS. So yeah, like, how crazy is that? We went from like cool, yeah, you could do like cool fly through videos. If you remember, that was what the early days of like the Luma app was to do the scan and now you can reanimate the camera and you let this thing render for 20 minutes and you got back something. Now you can literally take your scans and drop them into these real time environments. And it's like fricking amazing. I think on the left I'm getting like 400fps on an Nvidia GPU and on the right I've got this thing in Unreal Engine. And the cool part of it is, unlike photogrammetry and very similar to neural radiance fields, they still model these light transport effects. So again, imagine if this was a cardboard cutout model, you wouldn't have had all these light transport effects of the light going through the tree, et cetera. And so the way Gaussian splatting does this is by using this OG physics concept called spherical harmonics to model it. And so if you're trying to optimize stuff, you can get rid of some of these view dependent effects, as they're called view dependent, meaning as you change your view, the like materials look slightly different, but you basically get it all with Gaussian splatting. So I think it's super exciting. And, and yeah, like, you can do this stuff in the cloud, you can do this stuff on your fricking desktop. Now, like, I think Post Shot is a tool that not many people have used. But like, if you're working on a commercial thing and you don't want to upload your data, you know, with Luma's Terms of Service or Polycam's Terms of Service, like, you can train this all locally on your desktop with Post Shot with Nerf Studio, there's some of the models, Nerf Studio aren't commercial friendly. And then even in the phone in your pocket, right? Like, so if you've got an iPhone, like a modern iPhone, like, and you just want to know what 3D like, like what Radiance feels and reality capture is all about, just download the Scanniverse app and like, have at it.

So this is, this is maybe a dumb question, but like, so with like Nerfs and all this new tech, are you able to make like a really realistic 3D model of like a city like San Francisco? I mean, is that what you're showing me earlier? Or like, or is it only like a certain scene? Like, like, how hard is that?

Yeah, I mean, there's a bunch of new papers out, right? The initial radiance feels like. So there's a paper called Block Nerf that tries to scale nerfs up to cityscale using Waymo data sets. And similarly you're seeing in the Gaussian splatting world different papers about basically having like, you know, kind of like like nested hierarchies of splats that have really good transitions to model an entire like city and eventually the globe. So I think like that's the path that academia and industry is on and I think like already you're seeing city scale datasets that are very plausible in research and I think it's only a matter of time before that stuff gets into, into production.

You think that's like the future of Google Maps?

I think it's the future of maps for sure. Like you know, in immersive view there are certain indoor locations where you get a pre rendered neural radiance field that you can kind of like walk around and see. This is just the evolution of that. I think those data sets exist and there's like a handful of companies in the world that have it. So I think like that is the future of geospatial and like maps in general. But on the other hand I think what's interesting is like you know, this technology, like building a map of the world is easy, updating it is way harder.

Right?

Like when people talk about this like one to one digital twin of reality, it's like oh yeah, well like by the way, new stuff is built all the fricking time. Things change all the time. Seasonality is a fricking thing, right. Like so I think with this technology we've got and since we've commoditized capture because sensors are cheaper, compute is cheaper and now we've got access to the same sort of algorithms and approaches to model reality. I think like updating this model of the world is going to get a lot, lot easier. So I think like it's going to be very exciting or in the near future we're walking around, you know, driving around our cars and walking around with our like, you know, glasses or whatever and we're sort of updating this real time map of the world. I think we're very much on that trajectory and we're closer now than we've ever been.

What do you think are like the business applications to this tech?

I mean it's like all the applications that value stuff in the real world. There's utility and there's delight, right. I think like being able to. Not just like, I mean if you look at what Nvidia is doing with Earth too, right? We're talking about the physical structure of the world. You can think of like the Earth having very facets, right? Like there's like the sort of the terrain, like the natural like physical features of the Earth, then all the like, you know, human built things on top of that, the structure that we build. And then you can layer in like human activity on top of that, right? Like us moving around in the world, our sensors, our cars, et cetera. And then there's other phenomena like weather, right, like tides and things like that that need to be incorporated. So Earth 2 is this really interesting initiative by Nvidia to focus on the weather, like systems that govern like you know, basically day to day weather in the real world. And so if you've got that understanding of like, like the structure and geometry of a place where the sun is going to be, you can predict already things like, hey, can I install solar panels here? Like actually how much like sunlight would I get if I installed this configuration of panels? And when you layer weather on top of that, things get even more interesting. So to answer your question, I think there's a bunch of applications across utility and delight, like media and entertainment, obviously in gaming is like, I think the next GTA is absolutely going to be built in like a twin of the real world. Maybe this is the last GTA that we beat will be built by humans manually to emulate the real world. I think that's certainly exciting. That said, a bunch of games have already used reality capture, right, like from Call of duty to battlefront etc. But I think the utilitarian aspects are far, far more interesting. Whether you're like anything you're trying to do in the world of bits like from like building stuff to like disaster planning, like the range of applications is just immense.

Well even just, you know, one of the things that Jensen showed off at GTC this year was to create these sort of virtual worlds and then actually put virtual versions of like humanoid robots in these worlds and to sort of train them on this virtual twin of the real world so they know how to navigate the real world. And then once they get that training data, then they can sort of inject that training data into the real robots. So like this concept of creating this digital twin of the Earth will allow us to train a lot of these robots and machinery to operate within that digital twin before actually deploying it in the real world. To me, there's a lot of huge implications there.

110%. I mean like these, it's like a way of creating all the training data that these machines and perception models need to be able to navigate the world. Right? And what better ways, like you can create that like, you know, you can 3D scan a city block and then create all these different scenarios of human activity on top of that and feed that to, you know, like, and train like self driving cars, like, you know, self driving AI off of it. I think like the fact that we are, we've got a place where we can basically like we can teleport reality into the digital world and then also manifest the digital in the real world. It's like that bridge, I think is just very powerful for a bunch of different applications.

Well, I want, let's talk super quickly about ted. So first of all, congrats on even having a TED Talk. Like that's such an amazing accomplishment. Like, you know, some people say they've had a TED Talk, they really talk about a TEDx talk. And come on, come on, come on. You've actually given a real TED Talk a legit TED Talk. And not only that, but they asked you to host the TED AI podcast. So tell us a little bit about that and what's going on there. I mean, maybe share a little bit of your experience at TED and then, and then tell us about the TED AI podcast.

Yeah, sure. I mean certainly the TED Talk was a fun experience last year and I would say this year was even more fun. So I had the opportunity to co host session two, which was all about AI with Chris Anderson. And we had some amazing speakers like Vinod Khosla Fei Fei, the CEO of GitHub, Helen Toner export member OpenAI.

And even like, I don't know if you've checked her workout, but nice aunties like absolute trip down basically. Like to me what intergalactic social media looks like, it was a super, super fun experience. Yeah, I mean like, look, the T in TED is all about technology. And I think right now what's exciting is to put the like over time, TED grew to encompass, you know, not just technology, entertainment and design, but a plurality of topics. Right? And I think with AI sort of in technology being sort of this horizontal, like tech is a horizontal, but it's impacting so many different verticals in our daily lives, right? Like we can talk about all the applications, whether you're a creator, whether you're a knowledge worker, you know, whether you're a musician, you know, whether you're thinking about like national security and defense, whether you're thinking about relationships. And often in all of these sort of topics, you know, there's like a dichotomy that we as like Builders and consumers have to contend with. And so the idea of the TED AI show really is to outline those dichotomies and not necessarily take an opinion one way or the other, but sort of elaborate on the entire gamut of the good, bad and the ugly and sort of let people decide for themselves. And do that by talking to people from all walks of life. Like, people whose titles haven't even been invented yet, but obviously technologists, journalists, researchers, artists, the list goes on. And I'm just super grateful for the opportunity to be able to, you know, just bring my excitement into the space. Like, obviously, like, I want to build. Bring the lens of a creative. Like, that's like, built a following for over a million folks using these tools. But also as a product builder who shipped a bunch of this stuff. And then just, like, I would say, like, cautiously optimistic AI enthusiasts, I'm going into a bunch of these topics with those three lenses in mind, and it's just been a lot of fun. We've got some really cool episodes lined up for y', all, and I can't wait for y' all to check it out.

Do you have any idea of, like, you know, launch schedule? Like, is that. Are there dates planned out for it yet?

Yeah, yeah, totally. So May 21st, first episode drops, and then it's going to be weekly. There's going to be a little bit of a summer break there, but, yeah, 25 episodes in the season, and let me tell you, I think there's something for everyone.

Yeah, I do wonder, like, what was the general vibe at ted? Like, are people optimistic? Like, optimistic, or are they, like, really fearful of AI?

And if you look at TED also, you know, you had people like, like Bilavel. I know there were some other speakers there. I think maybe Mustafa Suleyman was there, or maybe that was a more recent one.

Yeah.

But then you also had guys like Gary Marcus, and I'm going to totally butcher his name.

Yudkowski.

Oh, yeah. Eliezer Yudowski.

Which are both more on the, like, hey, let's chill out on the AI side. So it seems like as from. From a speaker's front, it seemed like they had speakers on both sides of the arguments.

Definitely. I mean, the theme for this year was, like, the Brave and the Brilliant and, like, covering that gamut of opinions, I would say overall, the vibe is positive. So I, I, like, I, I'll give you a sample size. Like, I, I taught this discovery session, which was about the dichotomies of AI. It was about 50 people and sort of what we did is we looked at a bunch of these verticals in the AI space and, like, you know, essentially came up with, like, what happens if this goes really well? And what happens if this goes really poorly? And. And like, let's use chat GPT actually to come up with, like, a headline, like a pithy visual depiction of that desirable and undesirable future. And honestly, like, most folks in the room are optimistic about it, right? Like, but they're not blind to the downsides. I think, like, the problem with anything is extremes, right? And so, like, you know, you know, we're like. Nathan, I hear your concern of, like, if we're like, you know, oh, like, we can't have, like, doomerism and like, you know, and it's like, it's infectious. I think the same thing applies to the opposite narrative too, which is like, well, we just. Obviously we gotta keep, like, accelerating. We gotta keep shipping. I think it depends, right? Like, is sort of the boring answer to these things. And I think you can't understand the nuances unless you go dissect, like, that full gamut of, like, considerations. And so the goal is, like, really, like, the editorial perspective I'm trying to bring. And of course, Ted has a huge say in this too, is like, look, I'm 60% optimistic, 70% optimistic, which is not too dissimilar, I think. Matt, what you and I have talked about on most things, but I'm not going to be blind to all, like, the. The downsides of this stuff too, right? Like, and I think that's okay to say. And what I believe is just like, the speaker selection, you know, we're trying really hard to have a balanced perspective on the guests too, so that, like, what's. What I think is going to happen in the real world as well is like, you'll be able to hear both sides of that argument. Like, somebody who's, like, super stoked and thrilled on, like, AI Art and just thinks it's the bee's knees and it's totally cool to train on copyrighted material. Somebody who thinks like, that is the death of creativity as we know it too. And you have to.

Yeah, I think it's good to hear both sides. I mean, I agree, but I'm concerned. Like, I used to live in San Francisco and, like, they're, like, pushing for regulation now. Where, like, to.

Who's they? You mean. You mean Sam Altman?

The government? Yeah, well, no, the government. They're like. They're, like, pushing a bill through right now. They're trying to like they're trying to fast track it. I forgot who's doing the bill. But we're basically. If you launch a new large language model.

Need approval.

Yeah, you like need approval, but also like you have to basically sign something and like it's perjury if you're lying, that this model can do no harm.

It's like, yeah, like who's gonna do that? Yeah, yeah.

And it's. And so I agree, like, nuance is important and like, I do consider myself kind of part of the E. ACC movement, but more generally just like a techno optimist.

Do you have it in your body still? I don't know.

I don't, I don't. But you know, I like Beth, I like all the people who are part of that. It's cool. I think in general it's right, you know, and, but, but nuance is important. I agree.

Let me put it this way. I think the way I see it is like we've got enough talented humans out there that are like pushing for like acceleration and there's enough talented people out there that are pushing for, you know, I would say pumping the brakes for the lack of a better way to put it in certain areas. And I think like in totality, like we'll reach like some optimum solution because of those influences. And I think it's always been like that, right? Like, I mean, just like the early days of music, everyone was like, oh yeah, Napster. And like peer to peer and let's just go crazy. And then things settled down and we found a business model that worked. Maybe it's not perfect, right? Like people have a lot of gripes with the Apple and the Spotify business model, but I think we found this like globally optimum solution.

I mean, it doesn't always work out right though. They look at like nuclear plants, right? Like in the past the US Was going to build all these nuclear plants to solve energy problems. And we didn't do it because of regulations and because of fear. And now we're like trying to solve all these global warming problems. Other thing is like, well, we always kind of had nuclear there that we could have been using and it worked. So it doesn't always work out like it often does, but it doesn't always totally.

I mean there. One of the funnest topics is getting into how do regulatory, how do you politicians and regulators even regulate this sort of nebulous set of technologies? It's not just large language models, right? Like there's all the perception AI stuff and like, you know, the, the implications there, but like a group of these technologies that are like sort of permeate and oh my God, like some of the, the stuff that I'm like doing research right now on, on just the intersection of neuroscience and AI and what we're going to be able to do with just like, like passive neural, like interfaces with earbuds and things like that. I mean there's going to be some real big ethical quandaries that pop up. And so, yeah, yeah, trying really hard to bring like still be techno optimist but bring that balanced perspective and I think folks are going to like it.

I think, I think at the end of the day the important thing is empathy. Right? Like, the perspective I come from is I'm, I tend to be a very empathetic person. Right. I want to hear both sides of the story. I want to hear both perspectives and I want to be empathetic to both sides too. Like if, if somebody is genuinely worried that this technology is taking their job, I want to understand why. I want to understand, you know, what, what we can do to sort of mitigate the damages that could be done from this. Like, I'm always going to come from that place of empathy, which is why I've never sort of identified with the E ACC movement is like, I don't necessarily think we should always be pushing everything forward as fast as possible. I think we should be listening to the fears, we should be listening to the concerns. We should be figuring out sort of middle grounds. Like you mentioned, there's always people on both sides which kind of creates a decent checks and balance to make sure one side doesn't go too far and you know, AI nukes the world, but also the other side doesn't go too far and technology stops advancing completely. Those checks and balances, I think are a net positive overall. I think those need to be there.

I agree overall, but I mean, I think the big thing that was like, you know, the argument with EAC we'd be making is that, you know, compounding is one of the most important powers out there.

Right.

Like, and the idea that if we build faster than the technology in the future will be better and better and better and we'll start solving real world problems like cancer and all these other things that maybe we could have been doing if we weren't so like quick to just regulate everything.

Right.

And so I think with AI it's the same thing. Like, sure, like some regulation in the future might make sense, but if we just start throwing it out there right now, we're going to slow down the compounding and like the exponential, we, well stop the exponential from happening with our regulations. And yeah, maybe some jobs would be lost in the short term, but in the long term we could have cured cancer, we could have solved global warming issues, all kinds of other problems that we could have solved if we'd have just waited and see what happens with the technology. And yeah, now there's a big problem. Okay, maybe make a regulation but just don't do it right at the beginning like they're trying to right now.

I mean, India totally flipped their decision right? Initially they were like, oh, you have to get every model approved. And they're like, actually we're going to retract this model part, which was really interesting. And I mean like it's, it's like regulation also. Like it could end up in a place where it just only benefits the incumbent. Like the largest AI labs too, right? Like the regulatory capture like point of view. It's like, yeah, it could end up in a place where like any new innovation like from a startup can actually happen. And they're the ones that get these like onerous compliance requirements and they can't afford a team of lawyers unless they're like super VC backed. And then what an inefficient use of VC capital. Instead of innovating, you're sort of like navigating like the legal landscape of like a heavily regulated industry. So I think like your point about nuclear is well taken too. It's like, I mean like a lot of folks, I mean including Gary brings up the example of like, you know, like basically air travel and, and airplanes are so well regulated, but yet we've got this whole Boeing fiasco happening, right? It's like where you've got one really big incumbent. There's probably a revolving door between regulatory agencies and, and Boeing and you know, so it's, and when the Wright brothers.

Got started, I mean they weren't being heavily regulated as they were like inventing the plane. They were out there just like out in Ohio just you know, trying shit out. So I mean, I think one thing.

It'S a lot of people talk about, it's like just like technology sunsets, like regulations need to sunset rather than us adding up more and more regulation. So I think this is where things get geopolitical too. It's like, I think China is so much savvier about AI regulation than the US is right now. And I feel for the politicians, I think they're like asking for this type of engagement and I Think it'd be good if we engage with them on this and like bring those perspectives to bear rather than, I don't know, just being like, regulation bad, innovation good, let's keep innovating. And so it's like, it's nuanced. But then again, like, look, I'm, I'm.

I've always been a Libra and like, you know, kind of trying to build bridges between two worlds.

Yeah, no, I mean, I think the, this being a super nuanced conversation is like the understatement of the episode here. I think there's so many different like rabbit holes that we could potentially go down when it, when it comes to the regulation thing. I think you're going to have to be one of our sort of recurring guests maybe every few months, jump on and nerd out about this stuff. But you know, I, I do want to give you the opportunity to tell us what else you're working on. If there's a place you think people should go. Check you out your Twitter, your YouTube, obviously the Ted AI podcast coming out later in May.

Yeah. So just please follow me on Twitter bilavelsydu. You can also follow me on YouTube and TikTok at Billy effects. If you're interested in some more long form expositions that I do, check out the Creative Tech Digest. It's both a newsletter as well as a YouTube channel. And yeah, of course, check out the TED AI show. Maybe the one last thing I'll say is like if you're a founder and a builder in this space building with any of the technologies that we talked about and you're looking for early stage investment. I'm also a scout for a 16z games, so just hit me up on Twitter or you can email me. We'll put the email in the show notes as well. And I really appreciate you guys having me on. Wishing you all the success for your podcast. And Matt, I will see you at IO. And Nathan, I hope to see you in 3D sometime soon.

Oh, come out to Kyoto. Come on.

I got to make it happen.

Awesome.

Bill.

Well, it's been a blast. This has been one of my favorite conversations we've had so far.

So fun.

Excited to see you in person next week.

Cool. Cheers.