EP 244: Accelerate your GenAI journey with AWS

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

So many businesses are trying to figure out how they can accelerate their journey with generative AI. And you can look in all different directions and sometimes the more you look, the more confusing it might be. It seems like there's so much information, so many new pieces of software and services that we should be looking at. So today we're going to be talking with an industry insider on the correct way and one of the best ways that I think that you can accelerate your generative AI journey. Thank you for joining us. My name is Jordan Wilson and I'm the host of Everyday AI, where your daily livestream podcast and free daily newsletter helping everyday people like you and me not just learn generative AI, but how we can all actually leverage it. And if you are joining us on the podcast, thank you. As always, make sure to check out your show notes for more information. And if you're joining us live, you probably see something different. Yes, we are live here in person at Nvidia's GTC conference where our guest today AWS, is one of the big, big partner booths. I can see it from here. So, without further ado, very excited to introduce our guest for today. Shruti Kuparkar is the senior Product Manager at aws. Shruti, thank you for joining us.

Thank you. Thank you for having me, Jordan.

All right, can you tell us a little bit about what your role at AWS kind of is made up of?

Yeah, so my role is leading product marketing for X rated computing at aws. And that means basically helping our customers explore, evaluate and adopt X rated computing solutions powered by Nvidia GPUs as an example of to help power their AIML applications, their graphics applications, their high performance computing applications. Yeah, and so that's my role at AWS and for your listeners and hopefully a lot of them know about who AWS is, but AWS is Amazon Web Services. And so in simple terms, we are basically the cloud computing division of Amazon.

Yeah, and probably, yeah, probably anyone watching or listening this to this, whether you know it or not, AWS is probably involved in this process somewhere. Right. Depending on where you're listening, how you're listening, you're probably getting a lot of this through aws, you just may not know it. So, Shruti, I'd love to talk a little bit about how AWS shows up in generative AI. Right. Because a lot of People may not fully understand, you know, how big of a footprint AWS actually has in the generative AI landscape. And can you tell us a little bit about how AWS actually shows up in today's version of generative AI?

Absolutely. This is something that we get asked about by our customers as well because they are trying to figure out how to adopt generative AI, how to take advantage of this technology and get started quickly. What I would like to do is talk about AWS and what we are doing, but from the lens of four important considerations that we've identified through our conversations with customers, through our conversations with our internal experts. These are the four considerations that are important when getting started with generative AI. And so the first one is that there is no one foundation model that, that is going to rule the world, that is the right fit for every use case. And you know, again for the listeners, foundation models are these big, you know, the large language models, the LLMs. These are all foundation models that are pre trained on petabytes of data. And there are so many of them out there. There's llama 2 from Meta, there is Claude from Anthropic. You know, every week there is a new model.

Literally?

Yeah, I mean literally. Right. And so customers really need to identify which of those models are the right fit for their use case. For that we have a service called Amazon Bedrock, which makes a diverse set of foundation models available via a single API. It's just a simple API call and you can choose which model you want and test it out with the application you're building for your specific use cases. So that's sort of, you know, that's the easiest place to start, especially for developers because all they have to do is make a call to this API and they can get going. Now the second important consideration is that customers need to differentiate with their own data. Because if you think about it like all these models are available to everyone. So how do customers differentiate and gain competitive advantage? It's with their own data. And so this is where again we have managed services such as Amazon Bedrock or Amazon SageMaker, which allow our customers to customize their models, their applications on their own domain specific data. So think financial data or you know, legal tech, legal data in some cases healthcare and life sciences. That's a completely different modality of data. It's, you know, it's language but it's, it's the sort of the language of life through, through genetics and things like that. So allowing customers to fine tune and customize their models to their own data is Something we do really well and in a very private and secure manner. Because security, we, we often say security is job number one at AWS because as customers are trusting us with their applications and their workloads and their data, so, so we take security really, really seriously. And so that's sort of the second consideration. The third consideration is that customers may just want to use out of the box applications. Right? Like just like everyday people. Right. Like I use a lot of generative AI applications. I didn't build those, I'm just using those. There's lots of people using ChatGPT. One of my favorite applications is Perplexity. So similarly, our customers might also want just out of the box applications. And this is where we have Amazon codewhisperer, which is basically a coding assistant. It's your coding companion. It will help coders, developers write code, be much more effective and focus on the innovation and not the road task of writing the code. And then finally the fourth consideration and, and this is more applicable to people who are building generative AI pipelines with us. But it's important for everyday folks to know is that ultimately what powers all of this is reliable and scalable infrastructure. And AWS excels at this. We have infrastructure that delivers the highest performance while keeping costs as low as possible and helping customers achieve their goals in terms of the solutions, services or the applications they are trying to build. And Nvidia of course is a huge partner for us in this space. And that's actually the Compute X rated computing portfolio that I focus on. So. Yeah, so those are sort of, you know, those are a few ways in which we show up. There are, you know, AWS has 200 plus services, so, and honestly, each of them I would imagine, touch generative AI in one way or the other. So it's really hard to pick of my favorites. But, but these were some of the ones that came to mind. And I think that lens of like the four considerations maybe helps sort of make it a little bit easier to, to follow along.

Yeah. And you name some very well known, you know, large language models there. ChatGPT perplexity as an example, which is actually one of your customers. Right. So I'm, I'm, I'm wondering if you can kind of walk us through, you know, an example of maybe Perplexity or something like that of how AWS is actually powering that. Because I think a lot of our listeners, myself included, I use Perplexity every single day. So I'm even curious how does AWS accelerate as an example Perplexity's Journey in Gen AI. Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Genai. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on genie.

Yes, absolutely. Happy to share it. So Perplexity actually spoke at our flagship event, the AWS Re Invent event, last year and they shared their journey. So for all of you folks go check out that video. Honestly, that will do so much more justice than I can, but I'm happy to talk about it. So Perplexity again, for some of the listeners who may not know, it's basically like an alternative to a traditional search engine, right? So, so instead of when you search and when you want to learn about something, you right now have to go through many links and figure out which one of them has the right information. And all of that Perplexity has simplified it where you can ask the app a question and it'll come back to you with a really well structured answer with curated sources. And all of this is powered by their large language models. So how we help them is that they they've as an example, they've done many things. So again I'm just going one example they fine tuned models like llama 2 or Mistral which is these open source models that are available and they've fine tuned it on our P4D and P4D instances. These names sound really complicated. What they basically mean these are servers that are powered by AWS technology as well as the Nvidia GPUs. They use this accelerated computing to fine tune those models for their own application for what they were trying to do another service as an example that they used was Amazon SageMaker Hyperpod. Because when you are fine tuning or training these really large models, you have to do that you can't fit on one server. You have to do it across many, many, many nodes to be able to do it well in a way that where a particular node goes down, it needs to be brought back up really quickly or be replaced with something else. SageMaker Hyperboard makes that really easy. It makes this multi node training very resilient because it auto detects any failures it replaces with a new node. It makes the distribution easy, it optimizes performance. Those are just some ways in which we basically help our customers do sort of we take care of the heavy lifting so they can focus on their innovation and their use cases. But again, yeah, Perplexity. I just love the app. Use it so much and please go check out. I think their CEO spoke at Re Invent last year.

Yeah, yeah, no, and, and even kind of, you know, maybe we'll zoom in and then zoom out on this Perplexity example, right? It's, I think it's one of these startups talk about a being accelerated, right? It's gone from you know, launch to one of the most visited generative AI websites in the world in just about a year, give or take. But I'm sure a lot of that is being able to, to scale on aws. Can you talk a little bit about maybe especially, you know, I know we have a lot of people in startups and, and who work at, you know, now well funded startups in their, you know, series B, C, D, et cetera. Can you talk maybe a little bit about some of those other examples? And you talk so many different AWS services, you know that, that companies and enterprise can take, it can leverage to really grow. But maybe walk us through, you know, let's, let's say a Perplexity or another big company like that. How important is it to have something like AWS that you can start small and as you grow and as you grow you can instantly start using all of these other different services and scale. Can you kind of walk us through what that looks like?

Yeah, yeah, absolutely. So you know, when you talked about scaling, the first example that came to my mind is Adobe. Adobe is also a really important AWS customer and their VP of generative AI, Alexandra Costin also spoke at Re Invent. This is where I learned like I learned about our own stuff at our event. It's really great. It's such an educational event. So he spoke there and he talked about their journey on aws. And because Alexander's team had invested in machine learning for a long time and even before sort of generative AI became a term, they were already using some techniques to create tools for creators in Photoshop where people could have this generative fill or things like that. But then once, of course, some of these generative AI models came on the market, they double down on it. And Alexander talked about basically what he called building an AI superhighway, right, internally within Adobe, so that the infrastructure, the services are all in place so that his teams could innovate like he wanted to also take the heavy lifting off of his teams. And he used AWS for that, right? He used AWS to build that AI superhighway. And what is that superhighway? It's like I said, it's these servers at the very foundation, right? But then it's all of the other things like it's storage, it's networking, because where's the data living? It's living somewhere, it's in storage. You need storage solutions that can feed these models as fast as possible because they are processing data as fast as possible. And you don't want, GPUs are expensive, you don't want them to sit idle, you want them to be working. So you want to make sure that your storage solutions can feed, feed them data as fast as possible too. You also want really good networking, as I mentioned, because we have to distribute it across multiple workloads. So they use all of this, they use the compute services, the storage services, the networking services, many of our orchestration services, which is like, how do you make all of this work together, come together, work really well. So Adobe Firefly is a really, really great example and he also talked about how when they started out, know they thought like, oh, this is how much compute capacity we will need, right? Like because this is what, this is how much user response we would get. But it just went viral. People love the product, right? Creators love the product. Being able to just bring their ideas to life so quickly. And this is where the beauty of AWS comes in, is that once they realized that there were a lot more users they needed to serve, they needed to scale quickly and we worked closely with them. They I think 20x their capacity and all of it, or rather most of it anyway on GPUs and they deployed. The other thing that AWS obviously also does is provide a lot of optionality on all levels on the type of compute solutions you have. So we have GPU based solutions, but we also have our Own chips, Trainium and Infant Share and they use, for example, Adobe used Infant Share powered solutions as well. And then we have a lot of optionality on the storage side. We have a lot of optionality from a managed services perspective. You can use SageMaker or you can use, you know, you can build your own machine learning pipes, so to speak. So bringing it back a little bit, you know, to sort of the everyday application. It's like you said earlier, you may not even know, but AWS is powering it like so many creators today are using Photoshop. Another example is Leonardo AI. It's a startup outside of Australia and they are doing something very similar, image generation tools, generative tools for creators. And they are, you know, they're a smaller company but similar to Adobe, they're using a lot of our infrastructure as well.

You know, and even when you talk about generative AI, I think it's hard to keep up even for me, right? I talk about this every day. And every day you see a new, not quite a new Leonardo AI every day, but you see new AI image generators and new language models all the time. You know, I'm curious, you know, what is AWS doing or to kind of prepare for this, this influx, right? Because we talk about COMPUTE and we talk about not wanting, you know, all these, you know, GPUs to go to waste and to sit idly by. So you know, what, what is AWS working on or offering to kind of prepare for this, this influx because it is getting easier and easier, right, to build the, you know, generative AI software. So I'm sure you guys are seeing this, this nonstop, you know, influx of new customers. So what is AWS doing or to really prepare for this influx of generative AI companies wanting to grow and scale.

Right, right. Great question. And again, I think I will have to bring it back to sort of that. For people who are familiar with aws, they'll have seen this, what we like to call the three layer cake and at the very foundation, as I mentioned, is the infrastructure. And so in terms of preparing for that, our partnership with Nvidia is a really important piece of that, right? I mean we've heard the announcement at GTC this year with the Grace Blackwell 200 that's coming to AWS, that's going to definitely power a lot of generative AI innovation. We also do, like I said, continue to invest in our own silicon to offer more options, more optionality to optimize cost and performance. And so that's sort of what we are doing on that Infrastructure side, in addition to many innovations in the storage and networking piece. But then it's also going to be about trying to make it really easy to use. And this is again where Amazon Bedrock comes in. We also have actually this service called Amazon Party Rock, and I highly encourage all of your users to check it out. It is literally like it's basically a service through which anybody can build an app. You just have to go there and talk to it as if you are, as if you're talking to a friend and basically say, hey, this is the app I want to build. And just try it out, give it a shot. It's fun. It's so much. That's why we call it Party Rock. I highly encourage you and your listeners to check it out. Amazon Bedrock, of course, for developers who want to access foundation models through a single API. But check out Amazon Party Rock and have some fun with it. That's what we are doing. Party Rock is also a way for us to do that, to educate a larger set of people about this technology, about what it can do, and pull them in the orbit so that they feel empowered and then they feel ready to go to, say, a Bedrock and start building their own applications. Then finally, we talked about Amazon Q and Amazon codewhisperer. These are some of the applications where you don't need to do anything. They're already generative AI applications. You just need to use them. So you're just a user, but hey, they increase your productivity. They help you interact. Like Amazon Q, for example, can help you interact with your business data, can really help you even navigate AWS services. So those are sort of the three ways in which we are innovating across those three layers to help prepare for this influx. Because you're absolutely right. We see a lot of interest coming in.

And you know, you kind of talked earlier about, you know, at the AWS conference, you know, being able to, to learn from so many of, of your customers. And here we are at Nvidia gtc. And if you're listening on the podcast, you probably can't see this, but over here we have the entire exhibit hall, right? And we can see probably hundreds of trillions of dollars in market cap just sitting right there. And, you know, the future is, is real really being built just, just out this window that we kind of are overlooking here. Here, you know, as we talk about the, the GTC conference, you know, even for you personally, what's kind of catching your eyes, you know, specifically when it comes, you know, you know, to the future of accelerated Computing because there's, there's so much going on. But what's kind of catching your eye this week at the Nvidia GTC conference?

Yeah, that's, that's a really great question. Well, you know, so my background or just a little bit, you know, about myself, I worked at ARM for eight years. So for those of you who don't know, ARM is basically a really big company in the semiconductor space. They've pioneered the risk architecture, the ARM architecture that is basically in a way a competition to the intel grown x86 architecture. Anyway, point being, I come from this really hardware background and I am a big fan of Nvidia in that sense that their roots lie in that sense sort of chip design and you know, GPU design space as well. And so what is really has been exciting for me, not just about gtc but in general in this space. And it's not just Nvidia or even us with our own chips. It's the idea that a lot of this software innovation is interlocked now with the hardware innovation. Like there's so much synergy between what happens at your chip level, at your system level, you know, and what happens at your software stack and application level. End user experience. There's so much co dependence and synergy. Maybe it's always existed, but I feel like it's true a lot more. I mean, I was in a talk earlier where Jensen hosted the authors of the paper attention is all you need. These are basically. It was a paper that change the world, right? It was a paper where the transformer model, the transformer architecture that is the foundation for generative AI was proposed. And he just interviewed everyone and they joked about how, you know, Jensen was like, we are building the next GPU to be the size of your next model. And then the, the authors joke back, well, we are building the models to be the size of your gpu. So this sort of, you know, innovation which is bringing together the hardware architects with the, with the deep learning engineers with the machine learning scientists, like all of them coming together. That's just to me that, that synergy is very exciting to me. So I don't know if like that's one thing, I just, I, I guess that was a cop out. I'm basically saying it's all of it.

But yeah, yeah, and there's no doubt, you know, it's, there's always so much excitement, you know, at big events like this. But you know, we've talked about a lot, you know, when it comes to being able to accelerate your Generative AI Journey. You know, as we wrap up what's, what's maybe the important take home that you really want listeners and viewers to take away from our time here when it comes to really helping them leverage Generative AI in their journey.

Yeah, I think would my first thing would be learn about it, which they're already doing by listening to this podcast. Right. So I would say be curious, learn about it, start in small ways. Figure out what you can do today to learn more about it, to maybe try it out. You know, start using Perplexity, maybe go check out Party Dog that I just mentioned. So start small and then once you sort of figure out what are the right use cases. Now these could be use cases in your business applications. And, you know, you could suggest that to your colleagues. You could brainstorm, you could think about how you could advance those within your profession. Or it could be use cases in your own personal lives. You know, you could use, for example, like I use Perplexity all the time to learn, I mean, all of this stuff that I know and is a lot of it not related to aws, but in general related to the speed is through that app. Right. So you could, you could just identify use cases within your sort of personal life as well. I've, I've used Generative AI services to think about ideas for my child's birthday party. Like it's the, the, the opportunity is endless. So start small and then go from there.

I love it. And, and there's no, there's no greater advice because yeah, we, we covered so much here, but I love it. Just start small, go from there. Well, thank you for tuning in. Shruti, thank you so much for joining the Everyday AI show. We really appreciate your time.

Thank you again for having me, Jordan.

All right, and as a reminder, there's going to be a lot more, so make sure to go to your everydayai.com we talked about a lot. We're going to be recapping it all in our newsletter as we always do. So thanks for tuning in today.

We're.

We hope to see you back tomorrow and every day for more Everyday AI. Thanks, y'.

All. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.