#31: Deedy (Debarghya Das) - Indian Startups, AI & Search

Hey, Didi, welcome to show. Thanks for joining.

Absolute pleasure to be here, Nataraj.

So I found you via Twitter where I spend insane amounts of time like everybody else. But you're one of the few people I feel like the three things that intersect because if I look at my followers, most of them are non Indians or non Indian origin. And a lot of Indian origin or like people who work in tech are not that active. If they're active, they don't have huge followings and they're not strategic about and they don't have interesting opinions. And the ones who have interesting opinions are not actively on Twitter and writing about it. So I think you sort of like intersect all these three things. You work in tech, they worked in Google, now you're working at Glean and you write interesting threads. And I think one of the interesting parts and with this podcast also we try to do something is because of being Indian and because of being in venture, I see both US and Indian side of things. And I think you, your Twitter content is also somewhat mixed, taking writing about the Indian content and US content as well. So this, this is why I think you seemed interesting to me and I wanted to have a conversation with you on the podcast. But before getting into all that, can you just introduce yourself and, you know, what was your education and career till now?

Absolutely. Thanks, Nataraj. No, I, I appreciate that very much. I also do spend a little bit of time on Twitter these days. Let me introduce myself first. So my name is Debarho. I go by DD Usually you can find me on Twitter at Debargo, underscore Das. And, and in terms of my professional experience, you know, I. Let me start before that. Yeah, I grew up a little bit mixed. I was born in India, but I was, I spent seven years in the Bay Area when I was a child when my family was here, and I move in the fifth grade after and I spent middle school and high school in Calcutta and then I came back to America for college. I went to Cornell up in Ithaca and did my bachelor's and master's in computer science, after which I spent a year at Facebook. Didn't really like it, moved to Google and worked on Google Search for four years across New York. I spent some time in Israel and then Bangalore, and after my one and a half year stint in Bangalore, I decided to take the plunge and I joined Glean as one of the founding members. And Glean is now a unicorn startup SaaS company that sells enterprise search tools. So that's, that's my background So I.

Thought one of the topics I wanted to talk to you about was regarding Indian startups. I think you wrote a bunch of content around which companies are succeeding and which companies are not. So just give me an overview of what do you think is happening right now in the unicorn class, especially on Indian startups.

Yeah. So let me start by saying that when I moved to Bangalore, I was working at Google at the time. And one of the big reasons. There were many reasons, but one of the big reasons I moved back was to learn more about the tax ecosystem.

You moved from US to Bangalore?

Correct. I was in New York prior to that and I moved from New York to Bangalore. I knew it was going to be a temporary thing. It was just testing the waters. But I did want to get a better feel for how tech works and what's available in India now because I always hear so much hype about it from the Internet. You hear new startups are getting funded, there's over 100 unicorns in India. There's so many stories of big acquisitions. Shark Tank is now, you know, a household name in India. So, you know, to some sense, startups and venture and entrepreneurship has really become mainstream. And I wanted to see what the hype was all about. And when I was there, I learned a few interesting things. And maybe this comes back to my Twitter opinion is I like to have a nuanced take on things I don't want to have before or against something, but I want to understand what it's all about. And when I looked into the startup ecosystem in India, one thing I didn't find was there was no critique. Everyone always writes positive news about Indian startups pretty much across the board. Maybe a little bit of that has changed now, but that really irked me and that was kind of a negative signal. Everything can't be good with any ecosystem. There's some good, some bad. And the more I dug into it, the more of the bad I would find. I mean, most of the people writing these things are VCs and they have their incentivized to write good things. They're invested in these companies or people working at the start startups who are also invested in the ecosystem. So the more I looked into it, I realized, look, there's a lot of good stuff happening in India. Like, look, it's a growing market. There is growing disposable income. Technology, it's a technology first market. People are growing up with mobile phones in, you know, volumes that we, when I grew up in India was unprecedented when we were there. And that's great. I think there's a lot of potential there. However, a lot of the companies that I saw in India were just not real businesses. I think that's the first thing that struck me. You know, you are building amazing products. A lot of the products work really well. They have many, many users. They solve a big problem, need quickly. One of the great examples is Dunzo. Dunzo is a company for those of who, those who don't know. Dunzo is a company where you can order anything from anywhere immediately, including a free text box of custom instructions of what you want this person to do. And it costs you 30 rupees on top of, you know, what you would normally pay, which is incredible. But it's very evident that learning from some of these examples in America, some of these business models, I never saw them scaling. So then the more I dug into it, you see there are some insidious loops where these CEOs of these startups would raise, generate a bunch of hype, build a great product, raise a bunch of money, hire a bunch of people. And my goal, my question was, okay, well what's your exit plan here? Like these companies aren't going to make money, they're never going to be profitable. You're talking about 80, 90% loss margins. It's very hard to turn those kinds of businesses around in many of those cases. And I realized that what was happening is either they were too attracted to valuation and just didn't understand what building a good business meant because they were blown away by all the hype or they understood that at some point or they knew that going in and they decided to sell secondary. So they were getting rich and building their own personal brand. And if the company didn't work, it was like, oh, the market's not ready. But I tried my best, even though, you know, that's questionable. So that's sort of a high level overview of what the good and the bad is. And I think not every company is bad. But my opinion is a lot of, I would say aside from less than 10 companies, most consumer oriented startups in India, I struggle to see a path to profitability for them from an outsider looking in. And one last point I'll add there is more of a subtle thing is the reason you can critique the Indian ecosystem and this is great in comparison to the US is the mca. The Ministry of Corporate affairs in India asks startups and even private companies to file financials every year so you can actually see how much money they're making, how much money they're losing all of these details which you can't really get visibility in America.

Yeah, I think I've invested both in US and Indian companies. So one of the things that's, I mean there are bad things in India as well. But in terms of like regulation and filing paperwork, which is a lot more. Which includes this particular feature. Most of things are bugs. But there are some features in it, which is what you mentioned, where you can get every year you give the valuation. For example, if you invested in a US startup, you don't know what valuation it is running as an investor. Unless you're like a lead investor who's on the board and you get the insider information. But rest of them don't know which valuation at this point. Their internal valuation is for Indian startups. Every year you get an updated valuation. So you know very quickly what the feedback is from the company. But the point about incentives, right. A lot of startup media are startups. So your story is a startup like even though you can say TechCrunch in the US criticizes some of the companies, but mostly they're, you know, they talk about funding news and there's some, you know, occasionally they come up with a critique of company or like the coverage they do cover a critique made by different publication. But in India, your story or Inc42 and some other upcoming publications, they are pretty much startups themselves. They are like promising investors to attract attention. So it's like that incentive is not there for them. But there are other publications like I think this One Morning Context or something which is a purely subscription based. They did some investigative or some criticism about startup models. They're purely subscription based, I think. So they don't often get, you know, referred by your story or you know, Inc42 and that whole ecosystem which I get. And I think you're also right about the secondary share sales aspect of it which happened I think in US also.

Right.

For example, there was this company which was hopping at peak of pandemic. It was valued at 7 point something billion. The founder exited for $150 million. It is now valued again at I think 1.2 billion or something. So I mean in a sense the founder did the right thing. Like if you know the company is overvalued, like is it wrong that I am selling my stock? Right, right. That sort of applies if you know, in your public markets also. Right. If I am holding a stock and I know that it's overvalued, what do I do? I hold it or I sell it. Right. So there is Some interesting incentives and also a lot of early employees like look at all these things as the same. I think where the education really lacks is especially in Indian markets is a lot of people don't understand what is the incentive structure going in as an early employee or joining in as an engineer. Like how simple or how easily the whole system can break down in six months. That aspect I think is still pretty much unknown. Even though the top level, the founder sector people who are creating companies understand this incentive structure very well. And that's what happened. They raised at pretty high valuations in last two years. Even starting pre seed company with no theme, no product was raising at $10 million. And that might be a little bit normalized in the US from 7 to 10, but in India it's extremely high. Especially if you see things like what's happening on Shark Tank India or something, right? That's extremely high valuation for a pre seed company with no product, just an idea. But those things happen. Then we now see that there are no path into actually finding a product or product market fit or even surviving for next 18, 20, 44 months. And to your customer I think consumer products, right? I think there are very few categories large enough where consumer side of companies can actually work. And I think India is in a phase where the primary needs or like the absolute primary needs are winning. Like you take Swiggy for example, right? Swiggy is crushing it. And that's like basic need. It serves a very basic need. Done so also in a lot of ways serves a very basic need. And then there's like gaming and gambling which is working really well. And then I think there's whole ethic gamut with Vedantu and Byjus and which, which I think have lost the plot in terms of like what is the right scale and valuation. I think there is a use case, there's a right product market fit. But again they overvalued themselves, they overscale themselves. Because we've seen this in public markets as well in the US, right? Not all the SaaS companies that are overvalued are bad companies or bad products. It's just that you can't value something that is making 1 billion in 100 billion valuation. 1 billion is still a good business. Why can't you have a company which is, you know, 1 billion revenue and valued at 4 or 5 billion, that's a good business with 300 employees, right? But when you change that to value at 100 billion then your expectations of revenue are increasing and then you know, you basically revert back to me and that's what we are seeing. Right, but what are the some, you know, consumer companies that, that are not working well according to you? I think the numbers show it but some of the things that you are on top of your head.

Well, I'm going to name names but. And I've heard contrasting stories of this. You know I tweeted about this earlier and then a bunch of people got into my DMs about it. The ones that aren't working well in India. I know cred right now the financials don't look good. I've heard that their future predict forecast is great but everyone says that so you know, time will tell. You have companies like byju's that are, you know, just spending through the roof and they're not able to retain customers. All of their like sales expense is gone to waste because you get to book revenue the first time and then the customers churn and that's not particularly going great. Numbers aren't at the top of my head. In fact I would say given the current state of affairs. Of course things change. Startups grow. Numbers don't always look good at the early stage, but only like Zerodha Physics. I think Gamescraft and maybe a couple of gaming companies Dream 11 are the only four that I know that are consumer and that are I think sufficiently profitable. Which is, you know that's. I just made that up. Like I feel like they're actually profitable. The ones that are not sufficiently profitable is like Nika and Mama Earth are barely breaking even. They're just running a big, I mean they're great companies, good products, but they're running just big. Amazon, consumerized DTC brands. The other ones, other than these examples, I struggle to name many that are on track. You know, some at a scalar academy is not a unicorn yet but they seem on track to be profitable. Not many others are on a good financial track today that may change but not a lot. A lot of them are. But I do want to address some of the other points that you mentioned because you did mention a lot of them in response. I often get a lot of critique when people say hey, this is a very classic Indian retort is why are you pointing out Indian problems? This happens in America too. And to that my response is yes it does. There's enough coverage in America that I don't have to point them out firstly. Secondly, I do think that maybe I don't have conclusive evidence of this but in America I typically don't see as gist of loss margins, for example, like even when Uber was making loss for the longest time. And I think this quarter Q4, they're close to profit or almost profitable. Airbnb was profitable, I think for the first year since they existed. But they were never really losing 90% of their the money. They were losing a bit. But you could see a path for a lot of the Indian companies. I don't see a path. Maybe I'm wrong. I'm no investor, at least not in these companies. So I don't know. But I do want to say a couple of other things to your other points. You mentioned pre seed and seed valuations being high. Yeah, totally agree. Across the board. In America and India, they're high. Nothing to be done about that. A lot of the time they fail in terms of consumer need. I think that point is interesting because, you know, a lot of these companies have product market fit. That's not really the concern for me. At least when I was in India, Dunzo was amazing. There's this company called Bounce Scooter where you rent a scooter. Amazing as a consumer. Swiggy amazing. Zomato amazing. None of these products were bad. They're great products. The problems really were that India is a very price sensitive market. There are almost no brand retention or brand loyalty, unlike in America. And people will switch for like five rupees. You know, if something is five rupees cheaper on the other app, I'm going to order on the other app. That's sort of the mindset. And so without that brand retention, it becomes a bit of a fight to the bottom in terms of price. And I think that's in the business angle. A lot of these companies struggle as a result of that, because it's fair to say Swiggy is a great business. And I don't remember Swiggy's internal financial. So it's. This is not a dunk on them or anything. But Swiggy can't charge 100 rupees per delivery tomorrow and get away with it. They're going to lose market share if they start to do that. So they might be a fantastic product, but I don't know if that business is justified or how it will look in the future because of the price sensitivity. So that's kind of my view on the Indian consumer companies. Although I will say, you look at SaaS companies in India, there's not very many of them, but fantastic play when you're a SaaS company in India, on the other hand, you can make use of the Fact that talent is cheaper, even though it's rising talent is still way cheaper than America. You can build great SaaS tools and sell them in America and you can just undercut because your cost structures are so much better. And many companies are already doing this in various different ways.

Yeah, I think there's a leverage. I mean it's sort of like the opposite of what happened with the ID sector. Right. Because the entire IT generation where Infosys, you know, Tata consultancy, all these companies that came up, they got projects from the developed nations, from the US and the UK and they were able to execute them at the cheaper cost. Now you've sort of. But the profit is sort of shared with the US and UK companies and you are making marginal profits. But now if you're controlling the product completely and you at the same cost, you basically shift the economics completely on your side and you can basically make, you know, get all the higher. Basically you are the one who are, who's capturing the value essentially. So that's. And we have seen such success stories. I think Fresh Desk was one. What is the other company where the founder lives in a village or claims to be. Yeah, yeah. Zoho is another company which is sort of like the Microsoft of everything.

Right.

They offer pretty much everything and it's what it's. I think they never raised outside funding after a couple of rounds I think.

I don't think they ever raised. Yeah, I don't think they ever raised. And $1 billion revenue, I think, I'm not sure.

Yeah, yeah, I think they're at something some. I've heard a billion figure thrown around and they're now 20 year olds and I see them popping up in different companies for different use cases. Like one is using it for, you know, as a CRM tool. One is using for a different purpose. So they have a whole stack of tools. It's sort of like the whole theory. Like if you have a tool that is successful in the US they have a version of it. Like you take a. If you have Loom that is successful in the US they'll have a version of it and the US consumer can buy it. Right. You have Tweet Hunter in the US like they have a version of it. Zoho has a version of that. So you see any SaaS micro niche products successful in the US Soho will come up with that product pretty soon. So it looks like they will single handedly take that leverage that is coming up by shifting the product control to India. So I mean you mentioned Shark Tank. My sense is that you're following that. One of the interesting things in Shark Tank India versus Shark Tank Us that I've noticed is Shark Tank US is pretty much small and medium enterprise. There's always like if a Silicon Valley style pitch comes up like Mark even will throw them out. You know this kind of valuation doesn't work here. One thing I was amazed to see in this season particularly is it's much more closer to Indian startup ecosystem. I've actually seen couple of these pitches before they were on track. Tank like Flatheads was the shoe company which is similar to like a shoe company here there was another mind peers was like a mindful company that came on recently which I've seen before. And the interesting thing is it's much more similar to a venture ecosystem than the SME market that we are seeing. Even though we see a lot of SME companies but the language is pretty close to vc. Like no one says hey we are doing a pre seed on Shark Tank Us right? We are just asking for money and this is a small and medium business. What are your thoughts about Shark Tank India?

Well, to be honest my answer is going to be very disappointing because I don't watch enough. I get forwarded clips from time to time and then I see them and I know Ashneer Grover's iconic Dogla fun clip but no, I, I do notice all, I have noticed all of the things that you were saying though perhaps like not to sound terrible about this, perhaps that's bad. I, I wonder if India does have enough of a burgeoning small and medium business market. You know, I don't know, maybe they're all family businesses like they like that's most of them I know. But I do know that there aren't that many people in India starting non tech businesses and that's kind of the only comment I had. I know the Shark Tank India judges do face a lot of critique. A lot of people say hey you know all of your companies are loss making but, but no I haven't. The few things I've seen I find it pretty nice and entertaining and yeah, I have not much to add there. It's cool. I'm very glad that you know the terminology and just the fact that this is something that parents know about and everybody in India just knows about is exciting and it's a big change from like the case serials that people used to watch in the 90s, you know. So I, I think I welcome the change but I have no real thoughts on it to be honest.

Yeah, what I mean so you followed, you closely follow what's happening with Indian startups. We sort of saw similar trends in U S startups at at least in a growth stage startup level. I think the public markets played out pretty well. Like if all these companies. Right. Like all these consumer companies, if they were public I would see like that that's what you know, would happen with, you know what we have seen with some of the high growth startups, SaaS startups that we've seen in public markets here would have happened. So what do you think is happening with in general like big tech? And you know, I think there's, there's sort of like this narrative that hey, you know, tech is down, this, the golden era of 2010-2020 is over and you know, different thoughts floating around. So because of all the layoffs that we are seeing and the valuations going down in both public and private markets, I'm curious to know what your thoughts are on that.

Yeah, there's, there's a lot, there's a lot to unpack there. I think. Let me start with, I think the layoffs I'll go get to later. Let me start with what I think the future of tech is. A lot of the boom of technology that we saw before was sort of built on new devices and capturing user attention. If you think about sort of the biggest innovations of tech from 2000 to 2020, fundamentally a lot of them were okay, let's get mobile phones into everybody's hands cheaply, let's make laptops and hardware a lot cheaper and let's give them stuff to do. So you have utilities that will either help you do everything so search or maps or photos and what have you, cameras or you have, you know, social media websites where you can be entertained and connect with your friends and so on. So Instagram, Facebook, messaging for WhatsApp. And it's come to a point where this is what I think this is not really based off of too much like factual data, but we're coming to a point where we're a. The population is not going to keep increasing. Like that's probably something we all know now. The population will not continuously increase and people's time spent on their phone will eventually cap out. Like the incremental gains are very limited. So Facebook is at 3 billion total monthly users across all of their apps. Maybe that can increase a little bit more, but we're quickly coming to a point where it's going to reach an absolute saturation point. So then the question is if that is if we've peaked out on penetration, then the only lever that you can really dial is monetizing those users better. So there will be a lot of work on that. I still think we're very far from tech companies are very far from actually monetizing all of their users. Well, they're still using models like ads, which is in some sense a little bit low revenue per user. I feel like you can do a better job there. There's a lot of innovation to be had in monetization. But even all of those things combined, I'm not bearish. But I wonder where that new growth will come from. So a lot of people talk about AI and this and that, but sure, you can make AI and sell AI to other companies, but in terms of big deck and really saying where does the next hundred billion dollars of revenue come from? I don't know, Is it like a little bit can come from cloud, A little bit can come from get better monetization models. But will we ever see a 2000-2020 growth spurt again? I don't know. I don't see how yet. So that's one thing. That's where I would come from first. And then all of that sort of then trickles down to what actually happened on the ground, which is, okay, we had the pandemic that was a last, maybe some would say the last final breath of okay, now people are using their phones all the time because they're sitting at home. And so you get this huge spike in all sorts of businesses like food delivery and YouTube and ads all go up. But now that that's gone, what is the. What is the future revenues declined for all of these companies? They're definitely. I. A lot of people say they overhiered. I disagree with that statement because I'm like, they've always been hiring this way. They've been hiring this way since 2010. Pretty much every big tech company has been growing. They didn't really over hire the pandemic that much. But finally, you know, as the fat lady started singing the tech profits, tech revenue is no longer going up the same way it's been going up for 20 years. It was with 20%. 20%, 20% for almost every single company. Now it's not there anymore. So they've had to lay off people and actually focus on being a good business because you can't print money forever. And do I think that will change? I don't really know where it will come from, but it'll be interesting to see how new technology actually manages to monetize users. I do see that when we grew up, we would think of tech as a nice to have, like, okay, I'll spend 10 bucks on Netflix, I'll spend 5 bucks here, 10 bucks here. But we weren't spending real money on tech. You know, like, no one's spending a thousand dollars for something ever. So I wonder whether there are like premium companies or assets in tech that will be priced so high just in order to extract revenue from users and grow a business. There was never really that tier for consumer businesses before. So those are some thoughts.

Yeah, I think in the recent, like the launch of ChatGPT Bing interview that Satya did, one of the things he mentioned was like, yeah, if we as humanity have to go, like we have to increase our productivity, like, where does that productivity come from? And he thinks that AI will be that sort of underlying force that will shift all the boats and it's sort of like the, the rising tide that will shift all the ports. And I can see that play out. But again, how good that AI will become and how mainstream it will become and how faster it will become, like, those are all the things that yet to be seen. But there are, I think, I mean, I think we sort of like overestimate also, like how fast can we do this stuff? And sometimes we also over correct. I think in short term, I think what is happening, like not talking about the growth companies, right. If you just talk about like the big tech, the fourth or like maybe 10 companies at the top, I think these companies know much better in terms of what their expected revenue and profits are like estimated. They are much more accurate than a regular business. They're much more predictable. And I think there was this a little bit of a lacks of an environment which led that because everything was growing that I actually disagree with you there that they didn't overhead. I think they did over hire. And especially when, because I don't know if you've observed every CEO now in the earnings call says that, hey, we thought the pandemic was the normal. And I'm like, how can you think the pandemic is a normal? It is a pandemic. So I think there, there was some overheading. And I can understand for companies like, you know, Amazon, which had to dramatically scale their supply chain, it made sense. But I think pure software scaling. Maybe they did over hire. I mean, I don't know. Can we say that's engineering over hire or sales over hire or, you know, marketing overhead? And that has, you know, we can debate that, but I think based on like what I've seen like the Seattle company's growth, I think companies did over hire and also like overpay, honestly, because. Right. That also now comes to the question of like the let's talk about the growth companies.

Right?

Let's ignore the big tech, which actually are pro, you know, highly profitable. I think you wrote a tweet about what the four companies make, 1.5 trillion revenue or something. Right. These companies are making real money and like they have, you know, they're not worried about their business model. But if you talk about the companies like Coinbase, Snapchat and some of these companies which are in 10 to 20 billion market cap range, I think these companies really were not run well. I mean, because you can't hire engineers like Google if you don't have the margins like Google. I think for a long time a lot of startups taught themselves that, you know, we have to do this stuff and they didn't really think of the long term because if, yeah, Snapchat, you're cool. But if you're paying the same amount as Google or more without actually making profitable profits, I think that's mismanagement of companies. I think this will destroy the founder singularly owning the company status that the last 10 years gave. Because the last 10 years one of the things I think Google started this is the dual shareholder structure which enables Mark Zuckerberg to control Facebook entirely, which enables even Spiel to control Snapchat entirely. And now there's no really a controlling force. There's no force which can come in and say, hey, Snapshot is not well run because Snapshot is entirely controlled by one person's direction and even Coinbase for a lot of things. I think there was a Chief Product Manager, CPO they hired who left the company in three years, made 200 or some ridiculous amounts of money. Right. I, I will say that these are mismanagements that you know, we've seen because of, or like we can call it as a zero interest rate phenomena that we've seen primarily in not the big tech actually because like if you see at Microsoft and not because I work there, but actually they. It's not an extravagant workplace like people assume it is. But if you actually go into their buildings and you know, look at, you know, how the buildings are run and when you compare that with the Google, yeah, it's not extravagant. Google is a little bit more extravagant that everyone knows it's like the top of the bucket. But my actually observation here is actually the 10 to 20 and 10 to 50 billion companies were actually mismanaged, much more than the big tech companies. And big tech companies are just reacting to the fact that consumers are going to spend less. So let's just adjust as fast as we can and the predictability makes them adjust as fast as they can.

I have, I have some thoughts on that. So I mean I think that's, that's well said. Let me first respond to the offer hiring, let's say like the reason why I disagree with you there is and it's not really I think we're on the same page but we're saying the same thing in different ways. The reason I don't think it's over hiring is because the percentage they're used to hiring year over year hasn't changed. That's just factually true. Like it hasn't really changed. They didn't hire excessively more than they would have before Microsoft. Apple was growing at 5% a year. Google grows at 15% a year. I think Facebook was 25 or something like that. That is just, that was for every tech company, big tech company, just the recommendation your revenue grows by X, you hire by Y, which is completely in lockstep. Now the point where I do agree with you is now is that a good idea? Should you grow like that? Probably not. Like it's not, it's not wise to grow your engineers. Your engineers don't have to scale with your revenue. And that's mean that fundamental assumption is broken. So in that respect they over hired. But in that respect the pandemic has nothing to do with it. They've been doing that for a while. And I could argue that Google shouldn't have. You could argue, anyone could argue that after 2010 Google could have just stopped hiring. So there were times where that worked because everyone that they hired extra would go out and then create these fantastic product lines within the company that kept the company growing and had generated more revenue or really established the company's distribution or so on and so forth. That stopped happening because then the people became lazy and no new business lines have come out of Google for a while, for example. So that's my point with over hiring. You know, depending on how you break it down, I think there's slightly, we're saying the same thing but in slightly different ways when it comes to mismanagement and the market rate. Well, the hard part about what you're saying is unfortunately it's very hard to hire engineers at less than market. The reason that they pay them as much as they as Google and Microsoft and all of these other big tech companies do is because they won't come work for you for less. And that's really tough to combat. A lot of companies have done it and they've done interesting ways of optimizing the cost. Some companies have just decided we don't need A rate engineers to run our business. We can run our business with B rate engineers, but it'll be a fantastic business. So there's different angles to that attack problem. But it's not always easy, I would say in response to that, to hire at less than the, the market rate that a Google or Microsoft person definitely not gets paid.

It's definitely not easy. And there is a aspect of like paying a market rate, but I think that's where you as a company have to get a little bit more creative and like, do you increase stock compensation and reduce your base salary or. And it's not just engineering, right? It's not just engineering, right? It's across, it's, it's a product roles, it's sales roles, it's marketing roles, it's everywhere. It's not just engineering. And that's where I think companies like Snapchat and Coinbase really didn't do the way, for example, Amazon did, right? Amazon, when they were growing, they had to compete with Microsoft, but they found a way to not go the Google way. Amazon was competing with Microsoft, but it was not competing with Google. So Amazon was very, I think the principle of frugality, that somehow they cracked that combination of, you know, hiring young people, making sure they're staying for three years. They somehow had this, you know, a mix, a combination of things going for them and they kept it going consistently for 10 years in the same market right where Microsoft and Google existed and were able to continuously grow. And I mean they are one of the largest engineering teams and they also, I would argue slightly detailed in the last three to four years. And I think, you know, they went along with the pressures of the market in that sense. Last four years were crazy. But my point was like why are the early companies, at some point you can pay your, you know, founding engineering team at market pay salary and at some point after 50 or after 100, you have to realize and after going public that at some point you have to realize that your business model is not as good as Google's or you know, we don't have margins.

But my response to that would be that if you look at a lot of companies that hired that way, that was just the existing philosophy in the Valley and it Worked like Airbnb. They overpaid, they're fine now. Palantir didn't overpay. They're still fine now. You see Dropbox overpaid, they're fine now. So it's like you can play. You can see companies that overpaid and they worked out in the end. You can see companies that didn't overpay and they worked out in the end. It's really tough, I think in retrospect to go back and in hindsight just say it's very easy, I guess I would say to say if your business didn't work out, you shouldn't have paid as much and if your business worked out, great, good that you paid more. I just think it's a hard decision. So yes, in some sense, in any case where the company outcome is not good, you could say it's mismanagement. My only point is that it's not always obvious that that's what you should do because in both cases an outcome can be good. It's not always preemptively obvious that you shouldn't need that many people or pay them that much. My contestant, my actual view on this is I think you should pay more in, in most cases. This is obviously depends business to business, user use, case to use space. But the one that I enjoy the most is for highly innovative businesses, pay top of market and hire less. Because in most of these cases, people underestimate how much overhead it is to get work done. As you scale like 50 engineers who are excellent can get about as much work done as 100 engineers who are excellent. And then those incremental value add, when you add a bunch of excellent engineers is much less. The organization falls apart. And you can see examples of this even in stripe where stripe. In the early days they still hire very exceptional talent. But as they scaled, the talent's work was not being put to use because it was too much organizational overhead starting too many product lines. Those product lines aren't going anywhere. No executive is really managing these hundred different product lines. So that's when things start to fall apart more. So my, my view is for highly innovative companies, yeah, pay them and hire less and make them actually work and don't do, you know, your 20 hours a week like some Googlers do.

Yep. I guess shifting gears slightly, I mean you work on search at Facebook, I believe, and then you went to Google and now you're also doing a search company. Right. So I guess let's talk about if do these LLMs break search?

I, I let me Let me start with my spiel. That's a, that's a loaded, loaded question. I. It's obviously very hot right now and this is what everyone is talking about. I think that LLMs, clearly it's, it's abundantly obvious that for a large segment of tail queries, LLMs do way better than search could ever do. I think that is almost undeniable. There's no evidence to suggest that that's not true. However, there are certain classes of queries that LLMs don't really do a very good job on. Right. So if you look up fresh content we know LLMs don't do a good job on because it only trains up to a certain date. If you look up what's a good example, even old content that is not exploratory. So head type queries like, oh, tell me About World War II, LLMs are not going to do a great job of that. The web is way better as a resource to learn about something like that. So that, that's why, that's the first point. So I do think that it's not going to completely overhaul all of search. It does actually help a large set of very hard queries. It completely overhauls them. But the concern that I have for LLMs is much more than I think the. At this point of time, I was very amazed by them and then I sort of got nervous about certain things. And I'll tell you what my fear is. When you have LLMs enter search, there's a couple of things that happen. One is I don't think it's possible to keep your ad revenue up anymore. Right? So ad clicks that Google, for example, relies on to generate all of their revenue will go down. Because even though you can say I'll have citations and links under it, I don't think anyone's going to click that. So your ad revenue is definitely going to take a beating. Your cost structure goes up, so the business as a whole suffers. And that's, that's one problem. The second problem is if you think about the web ecosystem, this was a big discussion within Google, even when something called featured snippets came out. Internally called web Answers is when you start pulling content from websites and showing it on search. That has been done for a while, but it's been done carefully and bit by bit. And even that has sort of taken away incentives for publishers to make websites. So the web in general over time has really decayed. Like the web growth that you saw we saw in the 2010s is not the Same as the web growth today. And because there's no real incentive to create a website. Like if I'm a person, I'm way better off creating a TikTok video and getting some revenue off of that than trying to create a website for what? Like there's no incentive for me with LLMs that's even more exacerbated. So I do fear for what the consequences that that has on the actual web ecosystem. Because the research shows at least today that the only way, the only bottleneck to scale LLMs to better quality is to have more data, much, much more data than even the, even the web as we know it today. Where is that data going to come from if you know no one's creating anything? That's, that's point two. Third point is these citations that current LLM implementations on search do are already based off of a ranking algorithm that has been tuned and been worked on for a very long time. Now. Those ranking algorithms work off many signals, but one of the key signals is how people behave with web results. A lot of those signals are who's clicking where on what, how long are you hovering, how much are you scrolling, how long is the click duration when you click into something, etc. Etc. When you have an answer which just kind of muddles a bunch of results into one, do you get those signals anymore? And if you don't, then how are you expected to improve your ranking algorithms over time for things you haven't seen before? And what effects can that have? Like in the worst case, that can have effects like a grandfathering effect where nothing ever starts ranking again. Once you're ranked number one to five, you're always ranked number one to five because no other signals come in. So those are massive detrimental possibilities to the entire Internet and web ecosystem as we know it today. And that's, I think, a net very bad for humanity because the open web is actually the most beautiful thing about the Internet. Like no one likes closed silos and Instagram and TikTok. That just moves power from a democratized open Internet to these small companies who then control all their data data and do their individual stuff. Search doesn't work well, discovery doesn't work well. That's not a future eye that would excite me. So those are, that's what my concerns are. But I do think, yeah, it's a very groundbreaking new technology. It has a lot of potential.

What do you think about the fact that, you know, like let's. If you take a given topic, right, like take any topic, World War II or something and look at the real estate of entire world wide web. And if you train LLMs on everything on the web and what we are seeing right now in the early demos is that we'll get confident wrong answers because maybe in some topics web is completely wrong, right? And we've seen, I think, I don't know if you remember like for example when Facebook was in its most prime viral growth stage, it almost feels like everyone had a positive story about Facebook ignoring the fact that hey, a lie spreads around the world within two hours. So the truth takes 20 days. And then all of a sudden in 2016 everyone woke up and said hey, Facebook is the most stupidest thing we as humanity adopted. We went from one side of the pendulum to the other. And I feel like there is, there's some questions with LLMs which is, I mean if you control and then if you control the data that you are actually training LLMs that hey, then you're making an editorial choice that hey, right, this is what I'm going to train on the model or if, what if, what if the collective intelligence in web on a particular topic is basically, you know, non true or a conspiracy theory or some kind of nonsense and we're already seeing answers, you know, that actually point out that if you train on a large enough data. Because large doesn't mean it's true or correct, right? Large doesn't mean, just means it's replicated enough like it's true. Like a lot of web right now, you know, is pretty boring, bland and uninteresting because of partly search reasons. Right? Because everyone wants to rank first. Everyone is, you know, optimizing for SEO or not really for the topic itself. And then these different mirrored incentives developed. But we are still able to navigate it, it's still thriving. But there are corners of the web where this is exasperated than some other scenarios. And I think with LLMs also there are these scenarios that we will see more and more. Like once we will start playing around with all these integrations that are happening now, we'll see more and more that hey, web actually has some nonsensical content and majority nonsensical content. So now we have to like completely like block out some of these, you know, things on our models. I don't know what you think about that.

Well, I completely agree with large. That doesn't mean true. I always like to say that if you think a democracy works, just because 50% of a class says two plus two is five doesn't make two plus two five. You know, so the democracy doesn't isn't always right, it's just what the majority of people think. So that that's just empirically true. I do, I make a lot of analogies, obviously researchers are working on this problem right now. I make a lot of analogies to how search combated misinformation, Google search combated misinformation back in the day, or how Wikipedia is a very interesting example, very hard to reason about. Wikipedia also has the problem of combating misinformation information to make sure the truth is on Wikipedia and not lies. So taking the search example into, into picture, it's the, the difference between Google and Facebook is interesting because there is actually no incentive for a search system to be right there. You make more money by having more clicks, you don't make more money by being right. And, and so you kind of have to, you have to just be good and want to do the right thing and that's the only incentive. Sometimes the incentives can match, but in this case it very rarely does it match. People like reading conspiracy theories, people like knowing false things. Like you said, false news can spread in three hours and the truth can take 20 days. Search did a lot of work just out of the good of its own heart to try to combat this stuff. And I think when it comes to training LLMs, a lot of that same technology is going to be used in how you select training data. So it's almost like moving the information retrieval problem to the training data retrieval problem, which is okay, how do I now pick high quality training data in order to build a better model. And you already see some examples of this today. If you take image generation for example. I love this one. Dall e trained off a big fat data set of images and it's pretty good. But then you had mid journey we just made the choice to say we're just going to train our really artsy data. We don't care about the other images. Like images don't make any sense to us. We want art. And just by doing that sampling of the training data they create a different animal. And it's in my view so much cooler, so much more amazing that what it can do when you just select the training data. It's LLMs. I imagine they're going to do something similar to this. I don't think it's going to be perfect. It's very hard for humans to tell what the truth is sometimes. So I don't know how a computer would ever do it. It can only sort of use like algorithms to do it like, is this trustworthy? Is it a trustworthy source? Is this the best thing we know at this point? What is the time this was written? Because the truth can also change over time, as we know with COVID Like things people know think different things at different times. So what is the truth now versus the truth before hard problem? All of these. There are ways that these can be solved. But you're right, like I don't. My only thought is it's a very tough problem to solve. And it gets even more tough because once an LLM spits out an answer it, you don't know where it's coming from. You don't really have attribution to which part of it is false and why. So that's why, you know, obviously Bing does the search based LLM summarization thing and instead of just trying to spit out ChatGPT, like content directly so you can sort of curtail how much misinformation you have that way.

That also is an interesting, the attribution problem is an interesting one here because what LLMs are doing is they're synthesizing, right? They're not just directly pulling, which means you can't really attribute in the most accurate form. You can maybe attribute to the top three resources or something like that. And I don't know what the latest research on this is, but attribution is actually because one of the arguments, I was listening to a podcast where I think it was Jason Calacanis who was saying, you know, they have all this data from, you know, they're pulling from this website. Why can't they attribute? I, I don't think it's that simpler because it's not straight. Taking the text from a website and putting it out there, right? You're training on it and you're synthesizing it. And the classic example counterexample for that is if I read understand something and rewrite it, is it necessarily have to attribute each and everything, you know to particular source. I think that is an interesting battle that will play out for next couple of years, you know, in the courts to look out for.

And the way these algorithms work today is actually this is my, this is, this is, I think all of the big search engines do, the consumer search engines do. The attribution is you don't go to an LLM directly. You get your information retrieval, you make search happen first. You get results, you re rank them, you pick them, whatever you do, whatever you need to do, you find four or five results that you pick and then you Ask your LLM to summarize these results which may or may not introduce new facts. Some usually doesn't introduce new facts usually does a reasonable job, I think, of summarizing. And then you go down and do some post processing to say, okay, you've generated this blob of sentences from these results I gave you just now. How do I go sentence by sentence and figure out, oh, okay, this seems like it came from this one. This seems like it came from this one. That's not a real attribution and it's not really real generation either. You're asking it to summarize something and then you're attributing, which can have strange effects. So yeah, I agree with you. That's how it's done today. That part is somewhat attributable. It has mistakes. The part where it just generates from scratch. Yeah. How is that even different from how humans think? Right. Even when songs are written, they're inspired by other songs and you don't have to attribute everything. So where is the line there? The legal precedent is not there. I mean, every company that does large model training is dealing with it. Codex, the Microsoft, the OpenAI one for code that trains on all of GitHub that clearly violates GitHub licenses for certain open source projects. You have LLMs that what is fair use mean? As I've seen the same podcast, what does fair use mean when it comes to documents published on the web? And then you have the image generation case, which is like, okay, this is also clearly not to going cool, because these are images have copyrights and they're owned by the person who created that image, that artist. So each problem has different legal constraint. I'm not a lawyer and I can only imagine that the law is not set up for this kind of use case. So whatever decision it decides, they decide to make with the law would probably be like just lawyers being like, I think this is what the law says. There's no precedent for something like this.

Yeah, I don't know how many. I think There are probably 10 lawyers in the world who can understand both LLMs and this and come to, you know, come to a precedent on this. But moving on, like, let's talk about Twitter. Do you think, I mean, you, from what I know you have a strategy on writing on Twitter, you have a method to it, but do you feel like Elon is destroying Twitter and all your effort is going down the sink at some point?

Well, well, I, I will say that even though I have a method for it, there is Not a intent for my Twitter. I think that's an important distinction. A lot more people who are active on Twitter have a specific thing that they're interested in and that they post about. I still pretty much post about everything I'm interested in. So I don't have like, these are just my actual interests. So I don't have like a method to what I want to post about. Although I do ask myself the question, are you adding value here or are you just saying something that 100 other people have said? And if it's something that 100 other people have said, I just don't post about it. So, for example, cricket. I love cricket, but I have no new insight into cricket, so I will never post about that. So that's one. What do I think about? And actually the story of why I even started doing anything on Twitter is quite interesting. It's mostly in the last two, three years that I've really started doing anything. But as it comes to what, what I think of Elon running Twitter, well, I mean, Twitter for me is really. It doesn't make me any money, it doesn't win me any awards. It's not extra. Extremely valuable resource in my life. That's true. So that's cool, I guess. So. I, I like, I like using Twitter because I like hearing what people have to say and think about both my ideas or things that I've said and as well say that about other ideas. For me, it's like the most genuine social network. I detest Instagram. I don't want to see people's pictures or marriages. I want to. To talk about thoughts. That's like what I am as an engineer. So I like talking about opinions and thoughts. And this is where you can do that. What Elon's doing with the company is, look, I don't have a strong view on it. Like, he's bought the company and he has to make it profitable. It wasn't going to be profitable before somebody had to go and make it profitable. And the way he is choosing to do it is by firing engineers and trying to build the product back up in certain ways. And then some things are working and some things are not. I will say that I am generally very, very accepting of what he's doing because I know from like my time at Facebook that whenever you change a consumer product, everyone hates you no matter what. That's just how consumer products work. Change is bad. Like, nobody likes to see change on a consumer product, no matter how good the change is. And then they adapt. So that's one bugs and all the other stuff that's happening, sometimes it's tolerable. Like you're not going to turn off of Twitter because you know you couldn't follow somebody on one day that was a bug that existed. It's fine. You can tweet about it, you can make a huge scene about it. Great. It's not going to change the platform. The ad revenue going down I think is concerning. That has to be fixed. A lot of Elon's personal image and people's personal view of him affect their view of the product and the company, which I think is bad. I think the company should exist on its own and you run it however you want to. But don't mix the two necessarily. But that's my only thought. I am very highly in favor of quick experimentation and trying to build a better product. And I think that's what he's doing. And so fundamentally I don't see a reason to have an issue with that. And they're going to be missteps along the way. But hey, I've seen more innovation from Twitter in the last year than I've seen in the last 10, so I'm okay with that. I thought Twitter was doing nothing in the last 10. They launched a bunch of half assed features that never worked or never got traction. And so I'm very happy with what's happening now. I did think the API. Killing the API was a pretty bad move, I'll say that. So you know, that's. That, that's my take on it.

Yeah, I mean although one, I think a lot of people have this notion that Twitter in the last, I mean after Elon, what they're executing, I think that's a little bit misconstrued because last four years I think Twitter was executing pretty well. I mean at least in terms of like launching stuff and shutting it down. Even though they were like bad features but their delivery of was increased. They, I think they launched stories pretty fast. I mean fast in the sense for them, not actually when the industry was on stories and then they launched what is that, the Clubhouse replica, pretty fast spaces. I mean it never really again that we can argue with the product design but the end of like the previous version of Twitter was somehow was faster than the last 10 years. Which I want to give because I was feeling that, okay, Twitter like now introduced spaces as soon as like the Clubhouse said, you know, said they're not, they're not ready to get acquired. I mean imagine Clubhouse getting acquired by Twitter. That would have Been a crazy bull market exit for the founders.

It's worth noting that they had 8x the engineers to do it. So.

Yeah, of course. I mean, that's. That, that's true. I mean, as we've discussed. Right. I mean the incremental gain that you get after a certain point is, you know, minimal. And you don't get. I think there's also like philosophic, not philosophically, but there's an observation that humans, when organized above certain number, like you can't organize them, the whole dynamics change. I think the number often is claimed at 150. After 150, like everything changes. Like the same philosophy of management doesn't work the same way. You're arguing in teams doesn't work. So it's almost like even in venture, people see that, okay, once you have crossed 150, you have to introduce all the systems in places and, you know, basically treat your company completely differently than what it is before. Like 150. I mean, 150 is like someone came up with it and someone wrote in a bunch of books and everyone thinks that is the number.

Dunbar's number.

Yeah, yeah, Dunbar's number. Yeah. But so we're almost at the end of the conversation, so I'll ask the final question, which is, you know, what are the favorite resources? You know, YouTubers, podcasts, books. What is that? You consume that and probably unique or like niche content.

That, that, that's a good question. So I do like to. I, I'll say that I'm very cognizant. I try to be as cognizant as possible of not watching anything that's recommended to me. I just don't like that. I just feel personally bad if it's on the home screen, on a Netflix. I'm just like, everyone's watching this. I don't want to watch this.

There's a term for that, like naturally contrarian perhaps.

I didn't know that that was the term. So that's, that's how I find niche content in general because I just assume everything recommended to me is just not good. So, but in terms of specifics, I like to read a lot. I do read a lot of books. I don't think it's really tech books that as much. It's more of fiction reading. I enjoy that a lot. In terms of. I do not use Instagram at all much. I do do Twitter a bunch. And I think on Twitter you can get really far by going niche. So people with very like less than 10,000 followers have, have a Lot of content. It's very hard for me to think of names at the top of my head. But you can like go and see like my follow list and you'll see the niche people have a better voice because they're not afraid of consequences because they don't have as big an audience. But they will tweet regularly. So there's a nice little gap. Maxima of people who will have insightful things to say but will say it often that I like. Then you come to YouTube. Big YouTube user. I love many of the educational channels on. On YouTube that will break down. Like, I can't again. I can't think of too many. And there's a friend of mine that I really like. India in Pixels is a channel is run by a person called Ashris and he is awesome. They talk about linguistics in India, which I'm like, wow. The fact that anybody would come up with the channel about linguistics blows my mind. It's a very niche topic, but he just loves it, so he does it. And I think it's done phenomenally well in a way that, that, you know, normal people can actually go watch. So he'll talk about why different Indian languages evolved in different ways. And so that's one a lot of niche educational YouTubers. And then I would say when it comes to movies and stuff, I. I mean, I do watch some things that most people watch. It's not like a contrarian about everything. But documentaries in particular, I really enjoy watching. And then after I watch the documentary, I go and do like research about it and then understand what actually happened. Because documentaries can tell. They're incentivized to tell a narrative. So that's kind of in terms of content I consume. And then in terms of reading, I would say most of my read. There's a couple of sub stacks here and there, but there's none that I follow regularly. Mostly generic reading list. It's like hacker news, tech meme. And I will maybe read like stratatory, which is Ben Thompson's strategy blog about tech and podcasts. So I listen to a lot of podcasts and particularly Tim Ferriss and All in are my two top favorites. And that's kind of a gamut of everything, right?

Yeah.