Aditya Gautam: Understanding AI Agents Beyond the Hype

Business! Welcome to Go-to-Market, a podcast for RevOps sales and marketing leaders to dive deep into industry insights and best practices. Go-to-Market is sponsored by Fulcast. My name is Amy Cook and I am so excited to introduce you today Aditya Gotham, machine learning lead at Meta. Aditya, how are you?

I'm doing pretty good. Great to be on this podcast. How are you doing?

I'm doing well. Thank you so much for joining me. For the background from our listeners, it has been such a pleasure getting to know Aditya. We met at the AI Agent Conference that Firsthand Ventures sponsored last month in New York. And I was just absolutely stunned by the amount of information that was there. And the thing that was so exciting to me was to see people who are like really at the forefront, at the bleeding edge of technology, talking with each other about their presentations. And Aditya was one of the most outstanding presentations there. I was so excited to meet him and I was so, so excited that he was willing to come on the podcast and talk to me about AI. It's something, Aditya, that it is just a burning question for all of us. So first, before we get into some of the hot topics surrounding AI, can you tell me a little bit about your background?

Sure. I'm Aditya. I've been working at Meta for almost 3 years in ranking, recommendation, and LLM space. I've been in AI field for almost 10 years, and I've seen like how it has evolved from like normal one to neural network to LLM to AI agents, and then current multi-agent system that we are dealing with right now. And I also have experience working in Google as an ML engineer and computer vision for almost 3 and a half years. I did my bachelor's from Carnegie Mellon in, uh, specialization in information technology AI. Before that, I was working in Qualcomm for almost 4 years, so I made a switch there.

Yeah, absolutely. Well, that is, you know, that's quite a background, and it shows. Obviously, you're the technical machine learning lead at Meta and, you know, really doing some super interesting things with AI agents. So how would you describe AI agents in the, with, as it relates to LLM and with regard to, you know, the internet as a whole, how do you describe that?

Okay. I think a little bit of brief, brief history that we have to go into because right since last 2, 3 years, everyone is talking about LLM, AI agents. They're interacting a lot. Yeah. Uh, firsthand with that. So if you look back into the history of AI, the first big change that happened was a neural network era, which started from 2020, 2012 around with ImageNet competition. And a lot of advancement has been into neural network where the AI adoption actually started kicking up and everything. And there is this like paper to, into 2017 Transformer from Google Labs where The fundamental architecture has changed dramatically from fragmented approach. For example, for natural language, you have different architecture from computer vision, you have different. But now the underlying architecture has confined to this particular transformer block. That's where a dramatic shift happened in the industry. Then we have this BERT-based model, which are discriminative. That means they are finding positive, negative samples, or those kind of thing between different classes. And the another era change happened with the GPT coming in where rather than being a discriminative, you are generating the answers. You are generating images, not classifying images. You're not just doing sentiment analysis. You're generating these kind of content. So this is a little bit of shift. It's good to understand and have that as a background. And what is happening right now is that like with this LLMs and coming up and kind of like in impacting almost every industry through this like underlying common architecture. There are trends which are happening where the model is getting more powerful with more data and compute available.

Mm-hmm.

You are seeing there is a scaling law which is widely used in the industry where say that if you feed model more data, more compute power, more GPUs, it's gonna get better and better. So we, we are at like an inflection point where that is kind of like getting saturated or the benchmarks are getting saturated. So. Now the shift is happening more about like, how do we deploy this? How do we get economic value out of these LLMs or like these things, right? It's not just chatbot. Chatbot is good, but like, how do we solve like real problems with this? Like in industry, for example, financial world, like document processing, some classification or something more wide. So that's where the agent system is coming into picture where LLM as an individual where you are given input and output. The agents are using it in a more sophisticated way where LLM is just one part of it. They have different tools at their disposal. They can like call MCP protocols, which is developed by Anthropic. They can call different tools, your file system, search engine, like images from your Google Drive, and use that in a little bit of more sophisticated manner. So for example, like an agent can actually write an automated email depending on whether you wanna write a personal email. Professional email just by accessing your email, like Google Drive and other emails and other APIs. So that's where the value is coming into picture. So LLM is basically the brain. Agent is more like creating the actual value out of it. So that's where I was—

so interesting. I love— you said like a few things that I want to just rearticulate for our readers and our listeners both. So the first one is 2012. That's when the neural network started coming into play.

Yeah, that's where the first successful image classification competition was outperformed.

Okay. First successful image classification was performed. Was this by Google Labs?

This was by a bunch of people. A lot of people.

Okay.

There were like many pioneers, like in AI field who have been involved in this. But after that, like this neural network development just like took a pace.

Took, went like wildfire. 2012. Okay, next we go to 2017, and it's about— tell me about the transformer architecture again from Google Labs.

Yeah, so this is the transformer architecture where you have a self-attention mechanism where all the words are kind of like looked into each other and they create their own key-value space and, and kind of like understand like how the words are, associations are, their dependencies, and like different underlying characteristics, which was not possible before Transformer. So what Transformer does is, is created like this self-attention and feed-forward network. Feed-forward is there already in the neural network era. And if you block these back to back on the input and output, you see the model capability goes like dramatically up. So that's what like the model scaling is happening right now. No, not a lot of dramatic change in the architecture, it's just like putting more data put more layers of this transformer and see how it does.

So, okay, 2012, first image classification. 2017, transformer comes in and makes classification insane. Then 2023, ChatGPT.

Uh, yeah, ChatGPT, like the latest version came around 2022-23, but like there was ChatGPT-1 which was not publicly facing, but it was also a shift in the industry where you were seeing the generative values coming up.

I think that was more 2021-ish.

Yeah, you can say that.

ChatGPT, okay, 1.0. So that is very interesting because then you go from classification to content generation. And now one thing I learned from the agent conference is that we have really pretty much consumed the public data and we have created an infrastructure that is becoming systematized and regularized. And so Now the innovation is more on the data and how we're using LLMs to do what you said, build AI agents that solve real problems. Did I get that correct?

Yes.

Okay. So now we're going to talk about AI agents in the future and what you guys are doing at Meta and what you're doing personally. So now that you've kind of taken us through the history, what trends are you seeing?

So I think there are trends which are happening on the LLM side and on the agent side. So on the LLM side, what we are seeing is that the compute, there is a kind of like a saturation happening on the training side. We are seeing it's going to a test compute time. That means when the model is given the input, it is thinking, reasoning, considering what exactly is happening, reiterating over to that, having different kind of thought processes. So that is a proliferation of or, or the new knowledge delta that in terms of performance is coming on the LLM side. The other trend on LLM that is happening is the models are getting smaller and smaller also. That means there is a knowledge distillation happening from a bigger model and smaller models are learning it pretty efficiently with new techniques, quantizations and all. That means we would see a very powerful model down the line sitting in our phone and on devices. Which was not feasible like 2 years back because the focus was on scaling, being bigger, like more hungry model, feed them, get like the performance out. But now the trend is more like, okay, how do we make the smaller model more powerful? That's one thing.

Interesting. So now that you've really assembled this massive model, how do you distill it down into a smaller model to make it more powerful? Yes. It's more of like a, instead of a shotgun approach, it's a rifle. It's a very powerful rifle. Very cool. Interesting.

And on the agent side, what is happening is now there are like this whole new tools and the stack, which is coming on the agent side where if you want to have an agent or like want to have different agent do a particular purpose, you can do it like within an hour as a user. Like you don't need a tech expertise for that. So like there are different like multi-agent systems, open source and closed source, both are out there. So you can create an agent a lot. So there is a lot of emphasis on, or the industry is moving towards creating this agent system, creating the business value out of it. Mm-hmm. And kind of like just doing it a lot on, uh, this side. Tooling is one, evaluation is another one. And another trend which is happening, which is not like industry trend, which is more like government collaboration with industries, that a lot of emphasis is done on like risk associated and other things related to AI. AI governance is another hot topic among public policy people and everyone, where you wanna make sure that, uh, some of the fundamental characteristics of like as a country, democracy, and a lot of other things are still intact. And, and these, there are certain principles that AI models and whoever is using AI have to follow because I mean, you don't know where this is, this is a wild direction. So you wanna be like a little bit more, I mean, thoughtful about this particular thing. So EU is doing differently, US is doing differently, India and every country has just coming up with their own set of like laws and principles, like how how, when, like with respect to AI that is happening in the government.

Like, like no evil AIs.

Like, sorry, what?

No, like don't be evil type of stuff, right?

Yeah. I mean, like, uh, everyone has its own definition. So like the, there is a lot of work which is going on building new laws and policies on these kind of things on AI side. That makes sense.

For the AI ethics and governance. Interesting. So in terms of like, how do we make, how do we make this practical? How do we monetize this incredible investment in what we've seen and especially especially on the go-to-market side where I am, you know, I use LLMs every day because if I'm writing a press release or if I'm writing some content or if I'm doing some kind of manual tasks like that, it saves me hours and days. I see the value and I see the immediate monetization on the marketing side. What are some other areas where you're seeing like AI agents really providing a lot of value.

I think it's almost, almost every field. Try to— like, I'll put it in this way: wherever the humans are doing some— taking some input either from images or audio or like written text and doing certain job to put it into something else. For example, like financial analytics, like you're looking into stocks, analyzing what is the thing, what is the risk associated with it. You're looking to other sources and putting it into a format and like giving a rating of it, right? So This is something which can be easily automated by AI agents, but there are caveats like that, okay, these are sensitive industry, you have to have a human in loop and other aspects on that side. Similar to that, like one of the paper that I recently published was like on the how do we create a multi-agentic system architecture for misinformation mitigation? How do we detect misinformation? So if you look at the traditional way, it's a fact checker which takes into look into a piece of content, figure out what are the authentic sources we have outside. Understand, read it thoroughly, like multiple sources, like get, do a little bit of more dig, digging into that. And then they say, oh, this is actually misinformation. And then they classify this is a factually incorrect, or this is like a mis— like incorrect information or modified or whatever it is. Now these things can be done by different multi-agentic system where one agent is only responsible for finding the authentic sources. The other agent is responsible for developing the embedding where you can find similar top high quality content. And the third agent is basically looking into those. Piece of article and these authentic sources and ranking it, finding, providing, and doing a lot more sophisticated analysis. So I think application is there in almost every industry. Some are sensitive industries which you need to be very thoughtful about putting these agents in. So like, yeah, but apart from that, I think value is all throughout the stack.

That's amazing. It brings me back to the conference again where Another thing I learned was that building numerous separate AI agents to accomplish a set of complex tasks is more, is, is a better practice than trying to build one big AI agent to do multiple things.

Yes.

Can you tell, can you tell us why that is the case?

Yes. I think this is also a principle which has had a shift that happened on the distributor or the backend system, like going from monolithic to like a small microservices. This, this is similar to like, take any industry. For example, let's take a finance industry where you have a separate analyst which is specialized only in analysis of a particular thing. There are risk management department in any financial organization, and there are portfolio management departments, and the person responsible. And these people have a very confined and specialized knowledge and their expertise in one particular thing, right? You don't have one person doing all the things. Similar to that, like in agent taking, the agents are using different tools, technology, and they have their own prompt and specialization. Because of that, they are specialized in one particular field. For example, in the misinformation, like, there are agents— there is one agent which would only understand, like, how the web works, like, what are the authenticity, what are the different, like, in, like, different features and other things to give a score or a ranking to a website or a piece of article as a ground truth. There is an agent which understands and has the model specifically trained to see that given these particular good list of articles and this piece of content, which, which we want to classify as misinformation or not, like how good they are. So that agent is only specialized in classification. Other agent is used in understanding the web thing. So that's where you need to have a multi-agent architecture where the systems are coordinating and each of them are specialized in their own specific domain. And there is a centralized agent which is either looking into all the aspects and policy and whatnot, but That's where you have a fragmented approach, which is good also because tomorrow if you wanna make certain changes in one agent, you don't want to change any other agent. You want to just change the underlying LLM or tools or prompt of that agent and iterate over to it independently without affecting others. So, yeah.

That makes sense. Okay. That's very interesting and I love that we're seeing that. It makes me think of a real, team. And again, in marketing, you know, I am a generalist and I have very specific specialized skill sets on my team to deliver the best possible overall outcome. So I see where you're going with that. That's very interesting. Let's bring this back to monetization and let's say you've got a CEO that's very interested in using AI to monetize, optimize. And what would you suggest in terms of build versus buy? Should they build their own AI agent? Should, should they buy something? What would you say is a best practice?

Yeah, I think this would depend on industry and the company and the domain. It would widely differ, but there are certain principles which is applicable throughout industry. So for example, first, forget about like the AI and keep that as a black box and try to understand what are the Areas or the aspect in your workflow or organization that can get some value from AI, which can get some value from AI. First is like identify those things. Like what is the clear ROI metrics for that? Just for example, like, I mean, if you are like, there are certain companies who might be labeling or doing an documentation like overflow or analysis on that. Now this is, let's suppose if that is done currently by human, you have a high human cost, you have a certain level of errors, you need to have like a team of these people like working on different aspects. Now, like our hypothesis would be like, okay, the AI agent can do this job and this has been done in the industry and they have a decent level of confidence. So first would be like analyzing those hotspots where you have a high confidence or decent level of confidence on which can be replaced. What is the business value, savings, cost, and other thing that you will get out of it? And then it depends on the organization structure also. For example, like if you already have the ML engineers and data scientists in your organization who are very, very specialized, you probably would want to like built-in because you would have more flexibility on your models, your AI agents as compared to using something out-of-box. Also, it depends on the problem statement. For example, if there is an API or agent available which can do your job, then you don't really need to put any efforts outside of that. Just use that minimalistic. Start small, minimalistic approach and then make it more complex. Or like have in-house agent or see whether it's like— I mean, the, the whole way of building in-house would be that either you are compliance constrained, that you have to like keep your data in-house and the data centers, those kind of thing, and you cannot process this data external. That's a one industry or aspect where you have to do it in-house. The other one would be that like if, if your case is so complex, for example, if you're in a healthcare system thing. There are like HIPAA compliance, a lot of things, and healthcare data is different. That means you probably want to have your own fine-tuned models on your own label dataset to solve that problem. That's where you would want to build an in-house, estimate the cost, and see there is no solution out there that can solve that problem.

So yeah, so first step is to see if there's something someone's already built that will serve your needs. Makes sense, right? If so, it's going to be worth your time to— it's going to be worth it to just buy that without incurring all the development costs.

Exactly.

Yeah, that makes sense. If not, or if there's something that's very proprietary about what you're doing, or if you want to monetize it in the future and sell it to other people, obviously you would want to build it in-house.

Yeah.

Okay, awesome. That's a great, great suggestion. When it comes to user experience, we talk a lot about user experience. Especially with AI agents, right? They're getting better, but a lot of people just want to talk to people rather than, you know, bots. Sometimes they— people would rather talk to bots though, and the user experience is getting better and better. So how do you envision AI agents will be used to enhance user experience and even personalization at scale?

Yeah. So I think like shift that happened with LLMs is that earlier, like the AI and ML was like on taking a backseat. That means you, you are using YouTube and like Netflix and everything, and the algorithms are run by AI, right? So like it was hidden, but I mean, you were getting the output, but here you're directly interacting, right? You're directly having a chatbot agency. You're directly talking to AI. So that middle layer is gone. Now there is basically the AI has become in the front seat. Right. So that's where like the earlier it was implicit, now it is explicit in a way.

So we just pulled back the curtain and we're like, really? The person that's been running Netflix is a bot. Yeah. Yes. Interesting.

For a layman, this is like a big change, right?

Yeah.

For, for a, for a people who are like tech savvy, that's like, okay, this is just like additional layer, which is pretty great. So, um, uh, I think the user experience like here would change in a way because earlier. Let's say that like you can, you can talk like a human, like we are getting into that. You can explain the concept in a very detailed, like for example, like if you look at the chatbot, if you talk to a professor, they might not be able to explain it that well as compared to a chatbot, right? So the, it's incredibly smart. It is much smarter than human in a lot of different ways. Like one agent, you can talk about chemistry, biology, and about sports. right? Like you would not find a person which has that kind of thing. So the knowledge gap and basically the interface has changed a lot with these things because earlier if you want to learn about something, you have to do a Google search, look into YouTube video and put like, get from your own like report and other thing. But now you can have a condensed report and you can talk to your rec human like, explain me more in detail. So that has changed a lot. I think that brings a lot of like a different experience for human, right? And this also opens up a new field of like the AI and a human interaction. But earlier it was human-computer interaction, right? So now it's like a little bit different. It's like human-AI interaction is coming up as a new field. The other thing with the personalization is that like, I mean, all the things that you are seeing like using AI, they're all personalized, whether it's like your feed, ranking feed, like your Netflix, YouTube, whatever the, and social media and everything. Now the personalization would also happen in the LLM space also, like chatbot and AI agents. So for example, if you are a no, if, if, if you are basically a naive person in a particular field, then LLM would understand that from your chat history that this guy is not that expertise of a level. So I need to explain more in detail, right? As compared to a PhD in machine learning who knows a lot and they were like, okay, for the same prompt, the output would be drastically different and chatbots would eventually know that from your chat history. That this is the kind of response you like, this is the depth you like, you like more diagrams, less diagrams, like kind of thing. So yeah, that would also change with that.

Wow, that's crazy. An AI bot that understands audience and can personalize.

It understands. So you have already been like, we are already seeing this kind of personalization happening in the chatbot. So if, if you ask them like, okay, please provide a little bit by biography about me. So based on your chat, like what you have been doing, what kind of questions you have been asking, it will tell, oh, you have been interested in this, but as a side hobby you have been doing this. So this is, this is, this is where the personalization coming into picture, because personalization needs data. Yeah, it needs a lot of data. Like, you know, that's where you're seeing this value coming in.

So interesting. So, okay, so what, what are we looking at? What does the future look like in 3 years from now?

I think LLM is gonna be like a layer down in a way that like people would talk less about LLM. It's like given that there, these are different kind of brains which is empowering these things. The AI agents are even, there would be a layer above that that might come up. So you would be seeing like agents, multi-agent system working almost 24/7 at a very high performance and accuracy, almost like in different aspect in the field. And there might be some important critical security and risk aspect that we have to take into consideration that there are humans in loop also involved, making sure though they're doing their job, they're they're supposed to do, not like, I mean, just doing or, or basically gaming or basically like not following certain set of principles. That's, that's one aspect where even like I think a lot of government and everyone is paying a lot of attention to.

So yeah, interesting. Well, what advice do you have for CEOs, leadership who are looking to use AI agents or maybe who are suspicious of AI agents? What's your biggest piece of advice?

Oh, I think the people who want to adopt AI, first is just be very practical in which, like the cases where the AI can provide value, just not to go with the hype because that's what is happening. Okay. AI everywhere. Like people are like, okay, let's see where we can use AI. And then you see that like the total conversion from those prototyping to production. That rate has gone down. So having a good, a proper evaluation and a very practical understanding of where AI can provide value would be something very— the first and the most important thing to evaluate. Because implementing and doing those things with an AI agents or LLMs is not that hard of a job in today's world. Like, you know, you— if you have the right people, right mindset, and the structure and tech stack, you would be able to do that with a decent level of confidence. But like understanding and finding those values would be another like pretty important thing and being very practical about like percentage of conversion of the projects, which can actually bring value would be, yeah.

That's interesting. Well, clearly this, I mean, this is such an interesting conversation. I'd love to talk with you about this many, many, many more times. You're at the top of your game. How do you stay there? Unlike AI bots that can go 24/7, is that— we have to sleep and we have to recharge. So what do you do to stay at the top of your game?

I think this is like the industry has never moved this fast ever. Like if you look at like the spam of like non-LLM, so this is like a very fast moving space and things are changing pretty quickly over the week. It's hard to be stay like stay on the top of the game, to be honest, like because of the extremely high exponential, like rapid growth on the pace that we are in right now. But I think there are sources like what is happening from the major tech companies, like what is the tech blogs and what is coming from them. That's super important to look into industry. Apart from that, like what is the research conference and the like very practical conferences, like the one we went to, AI Agent Conference, another one. These are like very important ways and like to understand what is going on, what is coming out there apart from like the literature, which is already there, I mean, evolving pretty fast. For example, like the Google I/O, which happened like last week, it was tremendous. It's hard to process the pace at which like these things are changing, like their video audio model was like outstandingly good. Like, I mean, if you have been working on that enough, like you're probably like, it's obsolete already. Yeah.

You're celebrating. That's awesome. I love that. So you would say, you know, like to stay on the top of your game, like go to conferences, talk with people, you know, just learn as much as you can because this, this moment, right, is now. And if you're interested in AI, now's the time to get on board before you get left behind. Is that what I'm hearing you say? Yeah, I love it. Well, thank you so much, Aditya. I really, really appreciate you coming on. And it just to have someone of your caliber explain things about AI in these fundamental terms and helping us understand what we're looking at for the future is just so valuable. So thank you so much.

Thank you very much. Like, it's a pleasure to be a part of this podcast. And yeah, looking forward to more interaction in the future.

Thank you.