Ep 59 | How AI Actually Creates Value with Dhyey Mavani, LinkedIn ML Engineer

Hi, welcome to the new episode of Startup Strides. Today my guest is Vijay Mohwani. He is a machine learning engineer at LinkedIn. Welcome, Vijay.

Hi, Javed. Nice to be here.

Yeah, I'm really happy. I mean, also not just happy, but also I think it's our— after such a long time catching up properly, weirdly enough, on a podcast. We both went to same school and there was a year or two year difference between which we know are in class, if I'm not mistaken. But really happy to have you on. Yay. I'm a huge fan of your LinkedIn, all the things you post. I try to read them, get the AI updates, what's happening in the scene. And we're going to talk about a few of those things today. But before we, I guess, move on to those questions, I want to ask you about your Amherst time. We both went to Amherst. How was your experience there and how did it shape your thinking and career trajectory?

Yeah, for sure. I mean, Amherst was the first place that kind of made me feel like it was normal to be intense about ideas and still be kind of like a real person. So I came in thinking that I'd pick a lane early because that's how the education system was back in India. But Amherst basically refused that framing because of the inherent open curriculum. And the best part was actually just building this personal operating system of learning where you read closely, you argue clearly, and then you try to change your mind in public when you can. And you get comfortable kind of like being wrong in a room full of like smart people and like your professors and everyone's like really approachable and personal. So I mean, to be honest, it kind of shaped me in two big ways. So it first of all made me like much more interdisciplinary by default. So I ended up doing CS, math, stats, and that wasn't just because I wanted to kind of get all these three technical fields expertise, but I started off with math and stats and ended up realizing that for some simulations it's really good to have that. That CS background, and that's how my computer science interest got started. And then it eventually became like kind of like an intermix of all three culminating in my two theses and couple internship experiences that I've had. Also, Amherst kind of sponsored me throughout the entire experience, got me to experience multiple different styles of learning. And through like the study abroad experiences, be it at AIT Budapest or Columbia University, So it just made me go out of my comfort zone trying different things. And the second thing, it also like made me build and not just study, kind of help people on campus and beyond. So I spent a lot of time creating communities, shipping things, be it like leading an engineering for a student startup incubator at Amherst, which is called I2T, or building resources for the students as a peer career advisor at Loeb Center, or even just being a teaching assistant officially or unofficially for some classes across multiple disciplines. So. It just taught me overall to be like both theoretical and pragmatic about my day-to-day from multiple different perspectives. And I just really appreciate that because I feel like in the AI era, it's the core skill where, where like the tools can change weekly, but like the thinking habits and the perspective that you have, if you keep shifting it, then the habits are something that stay durable.

You know, you really made the most of your— I didn't even realize how much stuff you've done. I mean, I've heard of the things you've done, but you really made most of your time at Amherst. I actually had a follow-up question. Like, you sort of ended up doing 3 majors, you wrote like 2 theses. What made you do that? Like, you know, some people could have been like, okay, I'm just going to do 1 major and I'm just going to spend the rest of time doing some other things. What made you do and go after those Yeah, I mean, for sure.

So I think what motivated me really was the intersection of the three fields that I was like really interested in. And for the thesis part, like I wasn't even planning to do a thesis for any subject, just to be clear, but I ended up realizing these couple problems that my professors have faced in their day-to-day research. And I was like, okay. I want to help people, I want to like build something that can solve people's like real problems, then why not just do it for the researchers? Like for example, in my case, a mathematician and a statistician for my math and stats thesis. Talk to them about their day-to-day and their ideas on this and realize that they lacked this mathematical software or statistical software piece in their daily research work. And if I could use my thesis time as a time block to build this out for them, I think that would have lasting impact. And for sure they can like hire the research assistants down the road to extend it further and I can keep working with them. So I'm like right now still like working with both of the professors, just do incremental improvements now because most of the work is done.

Wow, that's fantastic. Do you think, so now that you look back at it in hindsight, do you think research was a good thing? Do you recommend people to do research?

Yeah, I think I recommend people to explore like their own curiosities, but also try to align them with some real problem. I think if you do that, then you have this inherent motivation and you meet up with the user or like the person that you're impacting in like a weekly fashion. And that just drives you instead of just going for like a far-fetched idea. Well, some people find. That really intriguing in a long horizon. But I feel like when you're starting out with research, it's better to do like applied research where you have like, um, practical time horizon and trackable, um, returns for like the users.

That's a very good perspective, actually. Never really thought about it that way. Things are changing though, right? Like with education, with work, with junior positions. AI is coming after a lot of things, and I think we still don't fully understand the extent of it, right? In the age of AI, sort of revolution, how do you think people should approach their education and shaping their trajectory?

Yeah, for sure. So I think education for sure is like shifting from like memorizing and performing in exams to kind of learning how to learn. And that gives you the real leverage and also kind of makes you really interdisciplinary when things change in terms of the jobs, the roles, or even your interests, right? So I like to break this down into like 3 main layers. So first of all, I think fundamentals still really matter a lot. There is a lot of hallucinations with AI, and you also want to kind of have basic level understanding of a lot of different fields to be able to pivot quickly as your interests shift or as the needs of the society shift. When AI makes like the surface level skills really cheap, Like so deep, deep foundations in like one specific area that you are like focusing on one or a couple areas and then general knowledge about everything else is really helpful. The second layer is like the fluency across multiple tools, right? And that's kind of becoming a mandatory requirement. You cannot say that I'm familiar with this tool that was released in 2024. If there is a new tool coming out or a version upgrade, you better be familiar with the features. Third thing is building a portfolio that proves that you can create value. Instead of just gathering the credentials, so to speak, because degrees don't matter as much as they used to. So if you're curious about something, just try to ship a project, try to build something or like build for someone. And I think there are a lot of opportunities to get mentorship as well. Learning by doing is another thing that's like really important.

Wow. This is really, it was really cool. Do you think liberal arts environment is good for that kind of thinking and approach to education? Do you think like, you know, I feel like when I look at education, I divide into two streams, right? Like we're hyper-specialized kind of, you know, you study, you get a technical degree, you get a, you know, business degree. And then there's more liberal arts environments that we both came from where like you, sure, you major in something, but you end up taking politics, psychology, you know, whatever. Do you think Like what kind of edge do you think liberal arts education provides?

I think the interoperability between the fields is something that's really important and is lacking in general from what I see in the workforce that is generally trained in the technical school, so to speak. So being able to make these connections across different disciplines or different, so to speak, job functions is really important because that actually unlocks real value that was hidden somewhere.

And you mentioned something I want to ask a quick question about. You said fluency in tools. It's a very interesting thing that there's so many tools out there, right? How do you manage that? How do you manage to be sort of staying fluent and learning while also working? Because there's a new tool being released like pretty much every day, or the old tools are being updated. How do you manage to maintains that fluency across so many different tools.

Yeah, I think that's, that's a problem that I'm also facing and everyone is facing. There's no like optimal solution for that, but how I think about it is trying to balance between exploration and going in depth in one tool. So try to vary your mileage in terms of the different tools that you're using day to day. But don't do it too much because otherwise you will lose that T-shaped edge that you're trying to build, right? So try to stick with a couple of tools for a couple of weeks and then try to pivot if you think something else is working better for you. But if you can try to like augment your day-to-day as much as possible through side projects outside your job, or even some tools that are like accessible to you at your job, thankfully enough, then that gives you kind of like this broad range perspective. For some of the tools, you don't need to like actually try them. You can learn from people people's experiences. Like, they have YouTube videos, people talk about the tools in their podcasts, or even just talking to people. Because I feel like talking to people, you learn a lot more than just like doing it yourself, because you are limited by the time you have.

I'm learning a lot by listening to you right now. Um, I want to bring the conversation towards your, um, your AI posts. You know, you work at LinkedIn, but also you right on LinkedIn, which is really cool. And I love following your posts. I think there are a lot of cool points you make there. Try to read and kind of stay up to date with the things that are happening. As somebody so entrenched in the space, I'm curious, what are the most exciting developments you're seeing right now and why they're so exciting?

Yeah, for sure. I personally see that agents are becoming more of a reality now. And people are trying to pay a lot more attention to how agents perform in the day-to-day work instead of just how impressive it is or like how smart it is. Because at the end of the day, if it doesn't unlock real value and the value can be here, like GDP, then it's not useful, right? Like it's just like a cool demo. So a lot of the effort I feel like right now is directed towards making evaluations for these agents really good, making the harnesses in which these agents are wrapped under really robust for the certain use cases. So recently GPT-5.2 came out, and the staggering fact about it was that earlier the GPT-5.1 was able to do 30% of the tasks better than a human through the GDP-VAL benchmark., but now GPT-5.2 is able to perform at a level where the humans are only able to do 30% of the task better than the AI. So if you think about the numbers from this perspective, it's pretty insane what's going on. While you see the other benchmarks, numbers are increased by 5 to 10 basis points in terms of the percentages. This one was like really staggering to me. So I feel like just staying updated with those kind of numbers coming out, but just then sharing my own perspective on how I view those numbers, figuring out how I interpret certain things that are being said in multiple events in SF, which is the hub for AI, is something that I'm really interested in. And that's what I keep doing. And I just try to share a couple of things that I learned from there, like my own perspective through LinkedIn once a day. And I think that's valuable because I feel like SF is not democratized enough and we should kind of try to disseminate knowledge as much as possible.

That's true. That's very fair. What I was curious about, you said you're kind of bullish on AI agents, right? And you're right, we're moving from an age where sort of AI was this new shiny object that everybody was willing to pay a lot of money for to a point where we're like, okay, we got it. It's AI. So what value are you getting from it, right? Because at the end of the day, everything is business and money, right? What's the biggest bottleneck you're seeing for autonomous agents right now? Is it the reliability? Is it the tooling incentives or the human trust? And I guess what kind of developments are you seeing in the space?

Yeah, for sure. So first of all is the latency, like the time it takes for the agent to get back to you with a certain work item, right? People think of latency generally from a median perspective, like the 50th percentile. 50th percentile query took, I don't know, like 3 seconds. But what really matters is that last mile or the 99th percentile that determines the user experience. Because think about it, like if ChatGPT always at times took a really long time to get back to you, you'll have a like really huge user churn, right? So that's something that's really important people are trying to focus on. As you can see in the industry, inference is becoming really important and the cost for inference is going down by 10-20x each year. So it's pretty insane there. I'll also say the messy human constraints are something that we should be thinking about a lot because privacy, compliance, trust, these things are not encoded explicitly into the models, or at least like not reliably enough. And there are surprising ways to inject different prompts into the models and get the information out. So figuring out a better way to do this or more reliable way to do this, even when the models are inherently probabilistic, is something that's really important down the road. I would say from a bottleneck perspective. All this concludes to be like reliability, right? If there is no reliability, a better tooling just means like you'll fail faster and it's not useful. And without reliability, the core thing, which is trust, that will not scale. And I think that is something that I personally feel is really important for the AI agents to actually unlock GDP value.

Very true. I feel like sometimes hallucinations are kind of downplayed, but for a lot of industries, even tiny bit of hallucination is a dramatic thing, you know. On a more macro level, do you think there are like certain, or maybe a single belief or multiple beliefs about AI that you think is wrong or incomplete?

I would say the belief that the models are basically solved and now it's just product UI or like application layer that matters. Is something where I'll definitely like push back because the reliability is not just like a UI problem or you just like running the model across multiple passes, like 3, 4 times as retries and getting to an answer. I think people are trying to focus a lot more in the industry now on benchmarks like FACTS to try to figure out how we can reduce the model hallucinations and make the models be grounded in facts. I think Groq does a really good job at this. Google's Gemini is trying to improve on this a lot. So I'd say like the wow demos, so to speak, are real. There are a lot of those coming along, but the gap between, as I said before, impressive product and something that's dependable where you don't need the human in the loop or you don't need the cognitive presence of a human in the loop at least is the entire game right now. The company or the product that solves that is going to unlock exponential value.

So do you, is it one of the reasons why, I guess what's harder than people think about, you know, when it comes to deploying AI at scale, is it, is this some of those things you just mentioned or some other things too?

Yeah, I would say one other thing is evaluation being like stopped after launch. In our, like in earlier, like AI models, so-called ML models, we used to have this evaluation pipeline or validation pipeline that used to happen. Before the launch. But now the thing is models drift, prompts drift, and the data that user is providing also drifts and their preferences also drift, right? So the users naturally in the last mile of delivery find edge cases and policy changes. So you want to make sure that your system is continuously adapting, it's able to handle these like systems of changes that happen in society on its own, because it's not scalable to manually override the prompts. It's just another bottleneck that you are adding. And at that point, it doesn't make sense to have AI agents deployed because you just added a lot more human labor to it. So I think that's something that's really important for scale. You need like continuous evaluations, canaries, like rollback strategies that are like really robust., and really in-depth observability into everything that's happening from the GPU layer to the application layer, how users are interacting with the AI agent through prompts so that you can try to figure out where the bottlenecks are and how to improve on it. Not just saying like, okay, this service wasn't connected and not just throwing like 500 errors on HTTP.

That's, that's very true. You, you said all the cool things and I guess taking everything into consideration, you know, there are a lot of AI use cases right now. There are new ones that are being made with every new startup, right? Which ones do you think are structurally durable versus trend-driven?

I would say a couple ones that I think are durable are RAG for enterprise knowledge or workflow support, because traditionally we used to have engineers or other people in other functions write workflows that are deterministic and complete these daily tasks or chores for the employees or for the company. But if we are able to automate even the creation or updates of these workflows through prompts, that would be great. And even taking it further, detecting the changes in policies through the RAG on enterprise knowledge base in real time. And updating the workflows without even a human in the loop would be something that will unlock huge amounts of value and just free up people to do more impactful stuff. And from a trend-driven perspective, I feel like fully autonomous general agents is something where I'm like really skeptical because it's really hard to build something that's like so general at this point when the models are not at their best potential and the models that are coming up are always gonna be just like getting better, right? So I would say the trend-driven ones are the ones that are not focusing on models being really good down the line, but trying to just plug them in and not take like a vertical position on what industry or what specific function they wanna focus on. So all the generic chatbots with like no grounding, or just vanity automation of something being cool but not really delivering value right off the bat is definitely trend-driven for me.

That's so true. One thing you mentioned about models, kind of, you know, it sounds like models just stopped evolving, right? They're going to continue evolving and we see new capabilities every day. What's your thinking, kind of sort of about this kind of a quick follow-up question more about the consumer layer?, right? Because I feel like sometimes you see an update and with that update, just a bunch of startups are essentially out of business because now OpenAI is solving it through their general model, right? How do you think about that? How do you think about the consumer layer AI startups?

Yeah, I think for the consumer layer, it's hard to create a moat for a long time, but I definitely think at least some of the current application layer companies will be absorbed into these like model labs, for example, OpenAI and Anthropic. But if you have a really good data formatting or like data collection mode that is happening under the hood for your current applications, then I think that can compound. And even if that gets integrated into OpenAI, you will have a really hefty acquisition offer from them. So it's really helpful and just like a strategy that will scale to have these data collection mechanisms, even though you don't really know what to do with all of the data right now, because there's this huge shortage of data in the world right now. And if you have really specific, reliable data on a certain vertical, then that will eventually down the road as models get really good, transition into, so to speak, a fine-tuned prompt or like an insight that you will add on to the model provided by these big model labs and deliver real value that the models in their base form cannot.

That's really cool and exciting in many ways. So I guess, are you seeing a difference in stability in terms of proprietary datasets or is there any other factor What does he also think of?

Yeah, I think proprietary datasets is one. Second is specific policies. So if you can figure out how to break down a really ambiguous problem into like a reward function for the model, try to break down the requirements and classify them into certain buckets that are gonna capture most of the randomness within the current datasets. So I personally feel it's really important to actually understand your data at this point because the understanding and insights that come from the data itself are gonna power the defensibility that you'll build on top of these models. And that oftentimes comes from traditional modeling techniques too. So it's really a misnomer to just disregard all the innovations that happened over the last decade in terms of time series machine learning, and just focus on the large language models and just the strategy of throwing everything at the large language model and assuming it will solve it for you.

Yeah, I feel like we have to run another episode at some point because I'm learning so much talking to you right now, and I feel like if you talk again in a few months, it's going to be a completely new conversation about new other Thanks. We're coming towards the end of our episode, so I'm gonna ask you my final questions. I guess one of them would be, you know, with all the kind of perspectives and the knowledge and information you are having right now, which, and if you had a lot of money right now, which AI companies would you be excited to back and you would be skeptical to back?

Interesting. Yeah, so there can be like multiple investments here. I mean, there are of course like different types of investors. I don't know if I've, if I was a VC archetype, then I'll have a different answer. If I was trading with public markets, I'll have a different answer. So I'll go with the VC archetype, given that your podcast is focused on that. So I would say thin wrappers don't really have a durable moat. And especially I think it depends a lot more on the team and how they think about their product. Because if it's just trying to capture a certain market through hype cycle, that's something I'm not too bullish on. Because as a VC, I want the company to become like a generational company down the road. And generally, if you look at it recently, PG, Paul Graham, said this as well. Generational companies aren't built on short-term hype cycles or ragebaiting on Twitter. Yeah. So that is something that I'll be like really skeptical of. Um, but personally I would say if someone is trying to solve really hard problems in an industry that doesn't currently have a lot of, um, AI enablement, but we'll see a lot of AI growth, then I think it would be a really good investment because at the same time the markets will grow in terms of the AI enablement in, in that specific field or industry. And the models of course are gonna get smarter, right? So if the team is great and these two factors align, then I think those companies would be really good investments.

And what kind of verticals are you thinking right now?

I'm thinking for sure like legal, for example, Udya, Harvey are really good ones. Healthcare be one of them? Healthcare definitely is one of them, but it's traditionally been really hard to solve these problems at scale. Like, we have seen a lot of revolutions in healthcare, but there's no— that there's no one company that has like stuck out. Same thing with the finance field, right? Again, like, you can argue it's the same with legal, but seems like the companies are doing a great job and trying to automate a lot of the work for the chief legal officers of the companies, or just in general creating an end-to-end firm. I mean, recently I think Jared Friedman from YC said this, that the first stack companies that are trying to build an AI lawyer are going to have a lot more advantage coming from the traditional world because they'll have this end-to-end data set as well alongside the perspective on different pieces of the experience. And then once the funding dries up or once there is a specific vertical focus that will enable a lot of like productivity for the lawyers, for so to speak, then they'll be able to double down on that specific thing and inform their decision based on everything that they've learned so far. Yeah, I think just people who are trying to build these ambitious ideas but still being really grounded on data is something that I like look for.

Fair enough. And I think, as you said, and I'm seeing it from my conversations, I'm seeing it across different domains, the full-stack approach seems to be the new kind of go-to way of looking at startups. Um, my final question to you would be, what question should more people be asking about AI but they're not asking?

Yeah, I'll kind of circle back to what I said in the last couple responses. The question that's really interesting is, what does the system do on its worst day, uh, and who pays for that failure because the customer shouldn't be paying for that failure, if anything. So most AI conversations right now kind of obsess over this median case, as I said before, median case capability. A lot of these benchmarks are also running on the best iteration of the model, which might not be the one that you will get in your computer when you like actually use it. So the real question is what are the known failure modes that we should prevent in production and how often they happen in production? Because when you're serving billions of users, or at least millions for the startups. You don't want this to happen without you noticing it. And on that point, how do you detect them, recover from them, or even prevent the recurrence of those situations? Finally, and is the business still worth it when you price in the trust loss or the support load that comes in compliance or even the reputational risk that these challenges face, right? So if you, if you really consider this, it's It's one problem where you are trying to answer the question of what— where is this reliability coming from? Is it from the model weights or from the product constraints that you built in or from the human oversight? Because whichever one it is, that is the actual moat and that's what you should be doubling down on.

Wow. This is— I love this episode. It was full of insights. It was great catching up with, uh, with you, my friend. We're going to run it back for sure at some point. You're going to make time for this again.

Thank you, Dey. Thanks for having me, Javed.

It's been fun.