EP 231: NVIDIA Making Money Moves: How they're using AI to change financial services - An Everyday AI chat with Malcolm deMayo and Jordan Wilson

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

Is Nvidia more than just chips? A lot of you probably, when you think of Nvidia, you probably think computer graphics, generative AI. But what you probably don't know is that because of all of that technology, Nvidia has its thumb on the heart of everything, including financial services, which is what we're going to be talking about today on Everyday AI. Thank you for joining us. Welcome. My name is Jordan Wilson. I am the host of Everyday AI, your daily livestream podcast and free daily newsletter helping everyday people learn generative AI and how they can leverage it to grow their companies and to grow their career. And if you're joining us on episode like 230 now, you're probably thinking this looks a little weird. This isn't how it normally is. Well, yes, this is actually our first in person interview for the live stream. We're extremely excited to be broadcasting live at Nvidia's GTC conference out here in San Jose, California. We're going to have a lot more this week, but today we are talking financial services and before we get into that, I do have to give a shout out because it's not too late. You can see still join the GTC conference for free. We're going to have the link in the comments and in our newsletter as well as a way to earn DLI credits as well. All right, so now after that big wind up, I want to introduce our guest, Malcolm demeo, the Vice president of Global Financial Services at Nvidia. Malcolm, thank you for joining the show.

Jordan, it's great to be here with Everyday AI. Really awesome. And it's awesome that you're here at gtc. So thank you. Oh, I'm excited.

It's been a, it's been a five year, a five year kind of delay of us just waiting back for this in person. Right. What's, what's been your vibe so far? Kind of walking around here.

We're sold out. It's unbelievable. So, you know, it's great. It's going to be an amazing week. So definitely. Audience tune in.

Yeah. So I mean, can you tell us a little bit about what you do in your role in global financial services?

Sure. Thank you. So most people think of Nvidia, like you said in your intro, as a GPU or a microprocessor. And we are so much more than that. We've been involved in financial services for over 15 years. My role as the global head of financial services at Nvidia is to make sure we have the right strategy, that we're building the right partnerships, that we're engaging with Lighthouse customers to what we call build our first bowl of soup, which is solving really hard problems with our accelerated compute platform. We sell nothing direct. We work through our ecosystem and we believe that everyone should win, starting with solving the customer's problem. The partners should be winning and we should be winning.

Yeah. Even going back to that, how does in the end, Nvidia actually play out in all of these partnerships? Can you explain that for our audience? Because is it just a platform that banks and financial institutions log into that helps them better use their data or, you know, what does it actually look like in video partnership in the long run?

Sure. So take a step back. If you go back to say, 2012, we helped JPMorgan Chase to accelerate their risk management trading platform, options trading platform. They saw an 80% reduction in total cost of ownership and a 40x speed up in their trading. Fast forward to today. AI exploded on the scene in 2022 with ChatGPT. And it's really hard for financial firms to leverage AI. There's a lot of work they have to do and they need help. So it requires ecosystem partners, it requires their current ISV partners, their software vendors to take our platform, embed it in so that they can deliver value to the customer without having to rip and replace their technology. And we deliver our platform through the hyperscalers. So all of the clouds are our biggest customers. They make our platform available to their customers. All of the large server OEMs, original equipment manufacturers like Hewlett Packard, Dell, Lenovo, Super Micro, I don't want to leave out, but all of them, got to get them all.

Yeah.

And they all deliver our platform and they have, they have expertise to help customers leverage, to help the financial firms leverage the capabilities in our platform. You mentioned dli, our deep learning institute, and it is a gem. It's sort of a hidden gem. DLI is free. It's free for anyone to come in and use to learn how to use our platform to build generative AI applications.

You know, Malcolm, you said something there that's, that's interesting. You know, you know, you talked about kind of this explosion of, of chat GPT, you know, onto the scene, you know, about two years ago now. How did you see that from your vantage point playing out specifically for financial institutions? Do you think it was maybe kind of watching from afar and Saying, okay, this isn't really for the financial sector. Or do you think that some, you know, banks and financial institutions were quick to adapt?

I think that today 2024, most people, and there's still some who are skeptical, you know, in the business there were a couple of technology waves like, like blockchain, all right, that had a huge hype cycle and still it's going to be a valuable technology, but it didn't deliver on the hype. And so there's a few doubting Thomases out there, but for the most part, everyone understands today that generative AI is a game changer. It is a tsunami wave size game changer through financial services for every industry, but in particular financial services. And when ChatGPT was announced, when generative eyes exploded on the scene in 2022, executives, not just the data scientists, who learn how to use the technology and use the technology to improve the efficiency, operational efficiency of these firms, to improve productivity and to create new revenue streams, in part partnership with the practitioners, the banking practitioners, the trading practitioners, the network payments firm practitioners, they, they started to come to see us. The executives started to come to see us because they want, they, you can, Anyone can use ChatGPT and see how powerful this technology is. And, and so they wanted to understand how can we use this in our business. So 2023, Jordan was the year of experimentation for financial services. How can we leverage this to generate better experiences for our customers? If you read any 10k today from a financial firm, one common theme is we need to improve customer experience and this technology is enabling that. In call centers with digital assistance for wealth managers, you have co pilots that are capable of being very smart in real time helpers. And the financial firms also want to be careful not to kill the next generation of bankers because a lot of young bankers coming into the business are in an apprenticeship. So using the technology required that experimentation and understanding. How are we going to use it in the firm this year? 2024 is the year of AI going into production and we're going to see some really exciting things.

You know, something, something that's interesting. There is, you know, even mentioning these partnerships that have been going back a very long time. And with artificial intelligence, it's not new to the financial sector. Right. You know, machine learning, deep learning has been pivotal for the industry for many decades. But with generative AI, it seems like kind of like what you were even saying. All of a sudden executives are paying attention and it's not necessarily, you know, people in IT or your, you know, chief technology officer Etc how should those in the financial sector be looking at generative AI specifically even when we're talking about large language models?

Yeah, well, so very they should pro approach it, I think with enthusiasm. And you want to embrace this technology. The, the reality is generative AI is very good and AI in general, pre regenerative AI was amazing at classifying information and we just do a ton of that in banking, a ton of classification, document classification, etc. Very good at making predictions, used to make predictions. But generative AI takes it kicks it up a notch and allows you to really for the first time analyze the vast mountain of unstructured data. And that's data, that document data, written data, that's video, that's images. So for the first time they have the opportunity to harvest all of this knowledge that exists in all the data. And the thing you need to keep in mind is that generative AI, while it's very good at analyzing and finding insights in all of this giant haystack of data, what it's not, it can't replicate the way humans make decisions. And we probably as humans make thousands of decisions a day and don't even realize it. And we just generally go through this decision tree of thinking, the logic, the way our brains are wired generation, that multi step thinking isn't there yet. So it's a great tool to help people understand. So think about a wealth manager. Instead of having to collect data from 15 different systems on their institutional client, Gen AI can do that for them, it can summarize it, it can do the extraction, but you still need that wealth manager to understand the customer's goals, to understand the risk tolerances and to build a really good strategy for them and plan and help them execute it.

So in your role at Nvidia, you've been able to see that this gen AI implementation in the financial sector from many different angles. I'm curious so far, not saying we're at a halfway point of gen AI implementation or anything like that, but how would you grade the financial sector right now in terms of keeping up or actually implementing generative AI? And maybe can you talk about just from your vantage point some of the challenges and triumphs that you've seen as well?

Sure. Well, so just to just to put a punctuation mark on the question, we're at the very beginning, we're not in the middle, and we're at the end of what you would consider general purpose compute. And we're at the very beginning of a new era of compute called accelerated compute. And what's driving that is generative AI. And so today or yesterday, financial firms have thousands of applications, hundreds or thousands of applications they've written. Tomorrow in the future that's all going to be done by AI and, and as an assistant to a developer and you'll have instead of thousands of applications that are hard coded and have and require developers to go in and rewrite and modify, AI is going to do all of that. So now you have the opportunity to do higher level things. And so, you know, from a grading perspective, I would say that financial services industry gets a very high grade, very, very good at adopting technology. This is what they've done for decades. They have to be careful because they're highly regulated. The data that they use is highly sensitive and they need to protect it. They are the custodians of their customer data and they do an amazing job at it. The, the uses of technology by bad guys is escalating and they fight them every day on behalf of their clients. So I would give them a very high mark.

Let's talk a little bit more about that because it's, it's very easy to see the wins for generative AI. You know, you gave the example, you know, someone having to look into multiple different areas for data and now generative AI can do that for them. What are some of the challenges specifically in the financial sector that people need to be aware of? Because sometimes people just want to just go out and leverage generative AI and figure it all out later. So what are some of those risks or things that people should be concerned about?

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies.

In the world do.

Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and Help get your team ahead and build a straight path to ROI on gen.

Well, that's kind of a two pronged question for financial firms. The challenges are this isn't easy to do and the technology is changing so fast that it's hard to keep up. And so working with an accelerated compute platform like Nvidia's takes that burden off their shoulders. We're keeping up. We make sure that every time a new model comes out, it's optimized to run on the platform form that we have financial firms coming to us and asking the question, how do I know if my model's good? How do I know if this thing's, you know, accurate? How do I, how do I measure that? And you know, the measure. The metrics for models today are things like, can it take the mcat, can it take the lsat, can it take the gmat, can it take the blah, blah, blah test? And that was a technical phrase, by the way.

Yeah, I failed that test.

So, and so, you know, they may want to show that the model is very, very accurate and can take the CPA exam. They may want to show that the model can take it can, can take the certified financial advisor test. And so we've built all of that, all of those evaluators into our platform. So you're going to hear all about that this week at Nvidia. That was a sneak preview from a risk perspective, you know, the biggest challenge. And so it's very hard to work with mountains of data. It's very hard to work with a model that's frozen in time. And so we've built a platform that helps them do that. So for example, the frozen in time, if you use an API service and you ask it a question, a current question, you'll get an answer back. Something like, I've only been trained through September 2023. Right. So we use a technique called retriever augmented generation. And this was created by Meta, but we baked that into our platform so that customers can now have a conversation with their most important data sources. Imagine just being able to ask a question in English and get an answer from your most important data source instead of having to sit down, write a structured code, test the code, et cetera. That's how this is changing the industry. From a risk perspective. There's the, there's the risk of hallucination. And so what we've done is built guardrails, and our guardrails ensure that a model stays on topic. It's not going to talk about religion, it's not going to talk about anything. Politics. It's not going to talk about anything mom taught you not to talk about. You know, we make sure that the model is only, only connecting to trusted sources. That's another guardrail. And the ability for customers to build their own guardrails using our open source platform is really. And they can do it again anywhere in cloud on prem through all of our partners.

Could you maybe just walk us through the whole process? Whether it's an actual example or theoretical of how financial institutions either are or can really push forward, save time, improve efficiencies by using Nvidia's platform?

Sure. A couple examples. First of all, the industry banking at its core is a batch system that was developed in the 1960s. These are mainframe based platforms and running Cobol, which is sort of your father, your grandfather's programming language. I did learn cobol. But at any rate, what we're able to do today is use an AI assistant to essentially analyze that code and write it in something modern like Java as a first step and then maybe a second step to something that's more modern like Python, et cetera, or C, which is very fast. So that's one example. And if you have code being generated, about 40% of the code in GitHub today is generated by AI. Think about it. If you're a financial firm and you have 10,000 software developers and they're now 40%, 50% more productive, all right, that means you can get 40, 50% more work done, that's more value to your clients, that's more capability for your clients to use. Another example is in the call center. You call your bank on Saturday morning or on lunch break during the week and you get the infamous Touch 1 for blank, Touch 2 for blank, Touch 3 for blank or wait for the next available assistant, right? Yeah, nobody likes that. Thirty minutes later a very nice person gets on and helps you with digital copilot. You're going to be able to have the option to say, you know what, I just have a simple question. If seven out of the 10 questions that are asked on a call center are just complaints, help me with something. All right, A digital, a digital avatar, a human, which we build in our platform. So a very lifelike person will come on and say, hi, how can I help you? I need to reset my password. Here's how you do it. Or I don't understand how to transfer money using Zell. Well, here's how you do it. All right. And so the opportunity to take those 30 minute wait cues down to you know, immediate, real time. And, and so what have you done? You've improved customer experience and the customer is going to walk away that one good experience. All the surveys say one good experience means they're going to use more of your products.

You know, yeah, we've, we've talked a lot about how Nvidia's advancement in this sector have changed things for the financial institutions or wealth management, et cetera, but we haven't talked too much about on the customer side or on the client side. So do you see that as maybe the future of where we're heading and instead of yeah, like waiting on the, waiting on the line forever, is there maybe just. I'm speaking with a digital twin avatar, you know, is, is it going to be that personalized even? You know, you talked about rag, right? Like would my even data be accessible with, with a digital avatar? Like would I be able to instantly make changes like that? Whether it's my personal banking, wealth management, etc, is that where we're going?

You're going to have AIs helping AIs. You're going to have AIs watch, watching AIs so that you know if the very first thing will be do you, do you actually need access to this information? Do you have the right authority to have access to this information? But yeah, think about today. Everything is so siloed. It's very hard for customers and bankers to have a converse, a holistic conversation and all that's going to change. We're desiloing AI is helping firms desilo. So it's going to be exciting.

So speaking of exciting, we're in a very exciting environment here at GTC in San Jose. What can you talk about where you're headed in the future? Because I know we're always going to get some big announcements and some things that I'm sure that this sector is going to be reacting to in the weeks and months to come. But what is in the near future for Nvidia, especially in your space, people.

Always wonder, what do you guys do? We started there and Jordan, the first thing to understand is that there's a lot of GPU and hardware accelerators. There's FPGAs, GPUs, ASICs. These are all hardware accelerators. There's lots of them. But there's only one accelerated compute platform and that's ours. And we started building this after AI's big bang. In 2012. Jensen saw the correlation between AI and compute and started investing heavily. So that when the world was surprised in 2022 with AI's iPhone Moment ChatGPT. We were ready. All right, most, most firms are trying to build what we have and what we're going, what you're going to continue to see from us is an amazing. In the last 10 years we've developed, we delivered a million X speed up to our customers and we're going to do the same thing in the next 10 years starting today at GTC. At 1 o' clock at Jensen's keynote. Don't miss it. He's going to be announcing so many cool things. And so the first observation is that we're a full stack platform and nobody else is. The second observation is that we're available everywhere and as we generate speed ups it makes this more affordable and more adoptable. And, and when we put it in cloud and you can use it as a pay for use, it means that more and more enterprises have access to it.

Is there a scenario in, in your mind when you look in the coming, you know, year or two, do you see just development going even, even faster? You know, I even think of my own, you know, personal experience with generative AI and I feel, you know, in the last like couple of months it's hard to keep up. So specifically even in the financial sector, you know, like we said, AI and machine learning, deep learning has been in play for a decade. Do you think and maybe with what's announced today, is it going to go into hyperdrive in the coming months and you know, maybe year ahead?

Well, the last three waves, sort of the Internet, mobile and cloud took 20 years to, to mature and, and to be fully adopted or, or largely adopted maybe the way to say it. And you know, compared to AI, that's glacier movement. Yeah, AI is moving at light speed. The announcement is every day there's another research paper published, there's another innovation, there's another. And so you need a company like Nvidia who is on top of this and building and who are our biggest customers? It's the, it's the clouds, it's the server providers who build all of this and make it, and make it available to the everyday person. So, so yeah, you're going to see in the trading business the half life of an alpha signal. Alpha signal is, there is the industry sort of description for above average profits or returns. All right, that the half life of a signal now continues to shrink because the market's moving so fast and everybody's competing in a world where they all have access to the same data. And so how they use that data, how, you know, is Is the differentiator how curious they're and how talented their, their team is and how expert they are at what they do and how good at using the platform we built they are is what's going to allow them to win.

What would you say, what's, what's your biggest piece of advice? Maybe whether it's, you know, people at extremely large firms who are still tackling, you know, gen implementation because it is an ongoing process. Right. What's, what's your best piece of advice for people working in large financial firms? And maybe they're going back and forth, maybe they're not on, you know, Nvidia's platform yet. But what's your best piece of advice for them to first understand generative AI in large models. But how should they be actually looking or what problems should they be trying to solve?

So first, first thing is embrace this and don't get left behind. The ship's leaving the dock, don't stand on the dock, embrace this. Lots of ways to do that. We have a launch pad, it's free. You can experiment on that launch pad for two weeks and try something out. Use our dli, our Deep Learning Institute to learn. Come visit us. Our executive briefing center is whirring right now. It is just jam packed. But when you leave, you will have a very good understanding. When customers walk in, they're like, we know this is for real, we just don't understand necessarily when they leave they're like, wow. So number one, embrace it. Number two, make the investments in your people upskilling them. And two, in the infrastructure. Because the infrastructure in everyone's data centers today is yesterday's infrastructure. They need an accelerated compute platform and you can start small and build as time goes. But over the next few years, they need to transform their data center from a data center, which is a cost and expense expense item, it stores emails, you know, to an AI factory which allows them to generate AIs that improve productivity, improve operational efficiency and, and help them grow revenue by doing a better job with the, with, in with delighting their customers, depending on what segment. So those are the three things. It's one, embrace this. Two, upskill your people and three, invest in the right technology.

Well, speaking of investing in the right technology, hopefully you're doing that with us today. Like we mentioned, Jensen's keynote coming up at 1pm Pacific. Malcolm Deo, Vice President of Global Financial Services at Nvidia. Thank you so much for joining us.

Great to be here, Jordan.

All right.

Hey, and as a reminder, stay tuned with us we're going to be here throughout the rest of the whole week with bringing you exclusive guests insider insights and also check out the show notes. We do have that giveaway going on so you can sign up for free to watch the conference virtually no matter where you are. Maybe you can't make it to San Jose. We'll be giving away that free GPU as well as the credits. So thank you for tuning in. Make sure to go to your everyday AI dot com, sign up for that free daily newsletter and we'll see you back tomorrow and the rest of this week at GTC for more Everyday AI. Thanks.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everyday AI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.