#288 How Generative AI is Transforming Finance with Andrew Reiskind, CDO at Mastercard

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

So when it came out, what it really started doing was taking data science and AI technology from the hands of the data scientists and democratizing it. And so I think that's really been the biggest impact. We hire our employees not as much for their hands as their brains, and that unlocks their brains. And that's what we want our employees to do, to think, to be creative. And it unlocks that the AI, because it consumes that much more data, that much faster, less human intervention. The quality of the data, the understandability of the data has become that much more important. Trust is our brand. We have no product without trust. If you do not trust using our products using your card or your device, the store, you're never going to use it and therefore we have no business.

Hello everyone, this is Adele, data evangelist and educator at datacamp. And if you're new here, Data Framed is a podcast in which we explore how individuals and organizations can succeed with data and AI. Financial services stands as one of the most data rich industries in the world. Every transaction, customer interaction and market movement generates vast amounts of data. This abundance of data makes finance a gold mine for generative AI use cases. So what are these use cases and how do you operationalize them and how do you ensure you're doing it responsibly? Enter Andrew Raiskind. Andrew serves as the Chief data officer for MasterCard. In his role, he oversees all of MasterCard's data in the iRemit. So there's probably no better person in this space to talk to than Andrew. Throughout the episode, we talk about Genai use cases in finance use cases. Mastercard is working on best practices in building generative AI use cases, how to navigate AI responsibly, and a lot more. Now onto today's episode. Andrew Ricekin, it's great to have you on the show.

Thank you Del, for having me.

Likewise. Thank you so much for coming. So we're speaking in 2025. It's January 10th today, over two years since the launch of ChatGPT kind of unleashed this generative AI hype that we're in right now. Maybe reflecting on the past two years, Andrew, how would you assess generative AI's impact on the financial services industry, both good and bad?

I would start from the fact that we had actually been using generative AI techniques for building some fraud models and for testing fraud models. And so for us, when the LLMs came out, which is just the latest version of using generative AI, we were used to thinking about generative AI, for example, we had created fake transaction data to test fraud models. And so for us, they were sort of, okay, this is great, this is nice, but as we know, there are always going to be challenges with adopting a new technology. And we're used to adopting new technology. So when it came out, what it really started doing was taking data science and AI technology from the hands of the data scientists and democratizing it. And so I think that's really been the biggest impact, not necessarily just even financial services, but for society as a whole. It is now very transparent to a larger population as to hear the potentialities of AI. AI had been in place and had been used by society financial services in many ways, but it was not transparent. I think having chatbots as an obvious example, image generation is another obvious example of where generative AI has gone, has made it much more transparent and available to users. And so within financial services, what it did was start sparking interest at all levels of an organization of how can I use it? How does this make my job better? What products can I create, what efficiencies can I get? So I think over the past two years, everybody's been asking lots of questions and doing adoptions in very different ways in financial services, but more holistically. And that is what we are really seeing. I think we're at the beginning of a very long journey. I would say that, Adele.

So definitely a long journey. And I like what you mentioned here on the democratization aspect because indeed, like generative AI and you know, data science and machine learning technologies in general have been part of the toolbox for quite a few years now for many organizations today. But it's interesting to see that ChatGPT has indeed democratized, put AI in the hands of the masses. And when you look at the latest generations of LLMs, you look at even commercial chatbots that you mentioned here, what capabilities have they unlocked for a company like MasterCard that simply wasn't there before?

The interesting thing I would just say is I don't know that it's unlocked anything that wasn't there before. Before. What it's done so far is create efficiencies and whether the efficiencies are in some of the personalization work that we do. So as a taking example, we have a Shopping Muse product for personalizing offerings on e commerce websites. We've done personalization using AI before, but it's just it now makes it that much easier, faster to take image to text, text to image, turn product catalogs into something searchable within an LLM. And that hadn't been there before. But we all did that. But we did that in different ways before. Same thing with the fraud tools that we built. We've been building fraud tools for over 20 years. This just made it easier to build some of them. Did it increase efficacy? Yes, but every year we've been increasing efficacy. Every year we're deploying new technology and new data to get there. So I don't know that my answer to you is yes. It unlocks something we were never able to do before. But what I would say is it's enabled a lot of efficiencies within the product development lifecycle, within actual deployment of products. And for our own teams, maybe focusing.

Here on the last part, on the own teams, how would you characterize usage of generative AI at MasterCard, even by frontline workers, folks who are in business functions thinking about commercial tools like Copilot or chatgpt or something along those lines. How is that maybe unlocked value?

It certainly made all of our lives easier. I would say that. And I'll give you easy examples. Right. Like many other companies, we've adopted it for coding purposes for the engineering teams in order to do coding. So that piece is been unlocked. The marketing teams for creating taglines, for creating images, we've unlocked that. We all have the chatbot available to us that can search a lot of our internal materials, summarize it. So that's unlocked efficiencies. One of the ones that I love is you can walk halfway in through a meeting and the chatbot will actually automatically just offer up. Can I summarize the meeting for you up to this point? And do you want to detail a specific piece you missed? It makes it okay to be late for meetings. What inefficiency that is might annoy your colleagues. But okay, so I would say those are the kind of tools that we've already adopted for those internal efficiency pieces, in addition to some of those product examples that I was laying out for you.

You know I saw you recently talking about unhyped or boring LLM use cases that drive value in financial services today. And I think it's worth barking here a bit. Give me examples of what you define to be an unhyped or a boring use case and what percentage of total use cases should they represent as part of a company's portfolio? A company like MasterCard for example. I'd love to see your perspective here on the unhyped.

It's these internally efficiency use cases like the one of note taking during meetings. The project and program managers are in love. I would just say that all of them couldn't wait to get their hands on this. Some of the ability to do desktop research has gone from 4 hours down to 11 hour. And so it's just all of those pieces that are making all of our employees their lives better, easier, allowing them to not have to do, as Mattel put it nicely, grunt work, but enabled to get the value out of them right. We hire our employees not as much for their hands as their brains and that unlocks their brains. And that's what we want our employees to do, to think, to be creative and it unlocks that. So I would say that as to portions of portfolios, I think that is a journey where most organizations are very much in that we are unlocking that internal efficiency first. That is the core, but that's shifting over time. The next layer are what are common processes across multiple organizations. So easy example is marketing is you're seeing a lot of tools out there to create marketing campaigns, to make marketing campaigns automated and get automated feedback on a regular basis. As those tools become more commercially available and we don't have to build them ourselves, that again expands what our portfolio is. The third portfolio is for, I would call it longer term investment, which is where on the beginning of that journey and we're deploying pieces of it like our product onboarding assistant that we've announced and that we're piloting out a chatbot for small businesses as two easy examples. That is slowly but surely growing and so would expect over time the high value piece being from that third bucket of net new products, much improved product offerings, easier to use product offerings, product offerings with better UX. That piece of the portfolio has got to expand. One would hope that becomes the 80% right now that's less than the 20%. I think that's a journey. I think from a person who's used to technology adoption over the course of my career, I would have said, oh, it's going to take Five years to get there. I don't know that that's the case. I think the rate of adoption, the speed with generative AI is increasing at a much faster rate than any of our expected. As we're seeing reasoning happening, as we're seeing the ability to do math happening, maybe that's within two years, our portfolio shifts. I think right now our perspective on this is we need to be nimble. We cannot have a set mindset with regard to this, and we have to be in a continual experimentation mode with regard to it and ready to adapt and ready to change.

It's fascinating. And as CDO of MasterCard, right, you're steering this massive data and AI ship. You've been steering it for a long time. You've mentioned that you've been working on generative AI use cases even before this hype cycle that we're in right now. And maybe I often think about what it means to be a CDO in November of 2022, when you're already in the thick of a data and AI transformation, and then ChatGPT is released, suddenly the board is asking you questions, what are we going to do about generative AI? So on and so forth. Given this backdrop, right, and what you just described now about that kind of portfolio of use cases, how do you as a cdo, keep focused on making sure that you're making the most of this transformation right now and that you're aligning teams with creating value with these technologies? What are some of the unique challenges of working with generative AI as opposed to other technologies that you've had to adopt?

Okay, so let me break that into multiple pieces. So as cdo, there are two things I would say. I'm responsible for data as a raw material, and then I'm also responsible for making sure it is used appropriately within the AI tools and it is fit for purpose for the AI tools. So, for example, I have data quality, data sourcing, and AI governance on that usability side of it. But I also just have data governance, data management and master data management, things like that. So when I look at it from the data perspective, I have two issues, one of which is I now have a new set of data of unstructured data that I now need to govern and think about how I govern. And the other piece is I have a new set of tools to do governance with. So on the new data assets to govern. Fortunately, we have structured data assets, structured unstructured data assets, or curated data assets. So taking it as example, we have many manuals and publications that we provide to our banks, issuers and acquirers in order for them to connect to our network. We're a network based business and most of our business is B2B. So I really have these curated unstructured data assets that I could deploy readily. I know how to manage. And so we are in a fortunate position because of that. Relative to the unstructured data assets, I would say a lot of companies probably aren't in that same position. You're really only dealing, let's just say with the emails, which good luck. What are good emails versus bad emails? I think that is a journey that even us, I mean it's to a lesser extent we need to go up relative to the tool set is I was just doing demos yesterday with new tools is how do I apply generative AI to my own data assets? To summarize it, to create the glossaries, to create categorization, to create classification. It makes life so much easier, right? I can speed things up. I can get to data assets so much faster. Right. You're hearing my excitement. This is really exciting for me that I can now improve that, which I do. I can change that. Some of my data governance teams, they don't have to manually classify things. No, no, no. I'm going to feed it to the generative AI. It's going to do a good job. So I would say that's my data side of the equation. On the AI side of it. I'm looking at it from that AI governance side and that usability side. And I would say an interesting exercise we went through earlier last year was where our priorities are. And the interesting piece is our priorities are still very much about data quality. The AI, because it consumes that much more data that much faster, less human intervention, the quality of the data, the understandability of the data has become that much more important. So it's just this feedback loop that we're getting that is just strengthening the need of just getting the basics right to feed the AI so that we know what the quality metrics are relative to completeness, to accuracy. That is mission critical. I would say the other piece to this though, on AI governance side though very much work in progress is where are standards for explainability? Where are standards for explainability and transparency and accuracy? English language is not an accurate language. We all talk past each other on a daily basis. That is just inherent in the English language and any other language, honestly. And so as we're doing this, our standards that we used to apply in AI governance have to change, have to transform the Tools that make accuracy analysis have to transform. I think you're probably aware from an accuracy perspective, people use QA pairs. That's not exactly quantitative analysis of accuracy. That's very qualitative. It doesn't make me feel comfortable as the Chief Data Officer. But I also understand that we're at the beginning stages there. So I would say that is a big thing on the AI governance front that we are looking at because we six years ago had launched our data responsibility principles and launched an AI governance program. And so we're very much used to putting AI governance on our tool sets. But right now it is very much relative to AI governance side of IT and launching a lot of these generative AI tools, very much still feeling our way through it, understanding it, learning, learning, working with others. We're happy to work with others in generative AI to build out the standards, build out the tools, work with new partners to build that out. But we're on the beginning of a very long journey on that. So I would say that's a piece that feeds to your challenge question.

What strikes me from what you're discussing is that there's a lot of foundational work that needs to be set in the governance, the quality, so on and so forth. But then there's also, sure, there's a lot of pressure on, okay, how do we make sure that we operationalize this technology in the wild and drive value from it? How do you balance that trade off?

That is my job. As I define my job. Historically it was innovating responsibly with data. Now it is innovating responsibly with data and AI. So my job function has expanded and there is a balance to this. And so just taking some of the things that we've piloted out there, and even the conversation was having last week is as we're using generative AI, it is, where is the testing to make sure that this works? Where are the controls to make sure that this works? I am not stopping anybody from using it. So just easy examples. We as an organization always made a decision. We're not blocking people from leveraging Genai tools. However, we are setting controls to say we're going to monitor what you're doing to make sure you're not putting any confidential information on them and you're not actually using them for products. So that was where we were when this all launched two years ago. And now slowly but surely, as we're launching products, it is a continual conversation with the product teams. So we actually have a couple things out in pilot in production, where we are saying, what are the controls? And as a B2B2C company, sometimes the controls are easier to implement. So as an example, we offer a lot of macroeconomic trend information out to our customers. One of them is called market trends about just to understand what is your market looking like as far as spending trends in specific categories among specific types of cardholders. And that one, we've taken that, which was static information and put it in behind a chatbot so somebody could do English language queries against it. We did lots of testing, Q and A pairs for accuracy, secondary models to monitor outputs. Rag right. We can name it all the controls we put in, but at the end of the day, we're testing it out with real customers. But the nice thing is those are banks, they're responsible parties. They understand that. They can call us and say, there's a problem here, they can test it. Great, let them test it. These are controlled places to do the testing. So I would say I want to create the new tools, I want to create those better tools, but let's do it slowly. Let's do it in that responsible way. Let's make sure the controls are in place. Let's make sure our partners are there to make sure that they understand their responsibility in this ecosystem because they want the benefit out of it too. Because to my point about democratization, it will democratize the benefits of generative AI for the bank's employees if their employees can get access to this information more broadly. And they don't just have to use data analytics people to get at it, so they understand the benefit for them. They are equal partners in this conversation. So we just use that as one example of how we're thinking through it, how we're deploying, deploying it, and how we're doing this in this responsible way.

What's interesting from what you're discussing here is the innovating and leveraging the partner ecosystem to also get feedback. Right. And connects back to what you mentioned about needing to stay nimble no matter how this technology evolves. And this actually connects to a report that I think your team put out on how to think about generative AI use cases across industries. And the categorization that you have is you have informed AI use cases, perceptive AI use cases, and proactive AI use cases. In a lot of ways, this reminded me of the different levels of analytics in the early days of data science. Can you expand on these different category use cases and how you think about them so informed?

I think the easiest way to think of it is it's simple chatbots. You are asking for information from these LLMs. Depending upon the subject, they become subject matter experts for you. Our use case is we're building out tools so that people can come to us just self service information about our security manual, our franchise requirements, so that they understand in a much simpler way. Here's information of how you need to work within our network requirements in our network instead of having to call up a customer service agent. So you could do this at two in the morning when you wake up thinking about a problem.

Right.

But also makes it more efficient for a customer and more efficient for us. So that is what we would otherwise call informed. Perceptive is multimodal. How do you take multiple kinds of information and feed them together to get to a more synergistic approach and a synergistic result? And so in our space, I think this is still very much in development. How you take for example structured data, which we have billions of, to put it nicely right, all of our transactional data and combine it with an English language kind of modality because math and English talk past each other much of the time. And so you have to think of this as a little multimodal. How do I do that to democratize structured data? How do I do it to come up with better analyses but also do this in other ways and shapes and forms? Because we've tried this before and with varying degrees of success. But how do you tie in images when you walk into a store with payment information in order to make it a better shopping experience? I will tell you, not something we did, but personal experience that I am absolutely loving. Is my supermarket instituted smart cart where you can go in and as you put items into your cart, it scans them and just tallies them. And at the end of the shopping experience you pretty much go through and get a QR code and you just pay off the QR code. And when there are questions, there's a tablet at the end and the customer service agent sees there's a video, so it's of your hands only, so it's totally nicely private that then can see what you put in, what you put out to do double checks. If there was a question at the end and I'm like, oh my, this is multimodal. The thing's smart enough to understand what I'm putting in, putting it out. It's got a scale. So it's also pulling in scale information. It clearly has product information going. This is a great experience of that perceptive AI from that Consumer experience standpoint, I'm sure on the back end store for the store, it creates a huge efficiency. So those are the kinds of consumer e commerce experiences that we want to power with a perceptive AI, but we're seeing examples of it. The next piece I would say is the proactive one. How do we take all of those experiences and create it so we actually guide you through or anybody guides you through where you want to go. Leveraging multiple data sets. So I think many of us are used to routing when you're driving to do some level of prediction as to where you should drive during rush hour, not just of where you are today. But imagine that with travel planning. I'm personally going on a trip to Mexico City for the first time to get to some warm weather and I would love for being able to pull in from my iPhone all these pictures of the kinds of things that I like to do and see. So it would be able to do that and then tie that into images from Mexico City of a consumer traveler experience to then give me feedback as here are suggested places you should go and by the way, here's how to route it because half of these museums are closed on Monday because typically museums are closed on Monday and here are your travel dates and here are the planes and pull that all together to give me those. Not just an itinerary but actually start booking in it and getting there. So I would say that is closer to where I think people are calling it agentic in certain ways, but to me it's a little broader than agentic because agentic could just still be within a single mode. But the idea here is to get you to a multimodal approach to like my point, it's about images. Images capture so much about me as an individual. What are the pictures that I'm taking and that's probably true of others and do that with text and with other sources of information.

And a lot of people are talking about 2025 being the year of agentic use cases and the year of agents. And it strikes me from what you're discussing here is similar to the different levels of analytics where you require some level of maturity along the spectrum to be able to operationalize at scale different levels of use cases. And I think majority of use cases I've seen today from most startups organizations lies in the perceptive and informed AI category. Majority informed, some perceptive. How long do you think we need to see maturity on the proactive side and the proactive use cases to start seeing real use cases that drive Value on these, on these categories, I think.

On informed we're already seeing the value. I think that is true. I think on the perceptive, again, we're already beginning to see the value. My shopping cart experience. So people are launching those kinds of products. I think for the proactive one, that's going to take a little time till we all trust how it works together. But that being said, this kind of goes to part of the concern of why people are slow to deploy because they're concerned about accuracy, they're concerned about reliability. There are all those levels of concerns that people rightfully have before you could actually deploy to market. But therefore are there use cases where you're going to have a human in the loop. And my bet is those are the ones where you are going to actually see faster things happen. Because I've seen some medical use cases that then give information to a doctor and the doctor is that subject matter expert who can then take that information and, and work with it. And to me that's a good example because you have a subject matter expert as a human in the loop to double check those kinds of outputs before actually deploying it. So I imagine we will start seeing those in the next year or two. I think true full on consumer experience stuff is more to the three to five year plan. But that being said, as a guy who's lived through various generations of technology deployment, this could happen sooner than what I'm used to from prior generations of technology.

You mentioned the human in the loop thing and I think that's actually quite central as a pillar right in how you roll out generative AI within the organization. You've mentioned those foundations of AI governance, data quality. But I think people and culture is also a major pillar in succeeding with these technological transformations. Maybe walk me through the cultural changes that need to occur and the skills transformation that needs to occur to be able to drive value at scale with generative AI in an organization like MasterCard.

I would say, and I will acknowledge this, that I feel very fortunate. MasterCard as an organization and everybody soaks this in. This is just part of our DNA. Trust is our brand. We have no product without trust. If you do not trust using our products, using your card or your device at a store, you're never going to use it. And therefore we have no business. So it really is a business imperative for us to have trust and engender trust. And therefore the AI governance conversation is an easy conversation to have within this organization because everybody understands, oh, I need trustworthy AI. And AI governance is the Methodology by which you get there, because it is just an approach and thinking about deploying the AI in the right way. Because it isn't just about compliance with laws. For us, that's baseline. It really is about that efficacy, the non discrimination kind of thinking. So that if, sorry, I grew up in the Bronx, if I'm in the Bronx, I don't get different answers than I am now that I live in Westchester, because there are different geodemographics from where I grew up than where I'm living now. And so those are important to us because we don't want those kind of negative impacts to populations. And so I would say for us culturally, that was an easy thing for us to implement six years ago, the AI governance piece. And people are used to, are thinking about it and are doing it so that now with generative AI, people get it and they understand those concerns. I think part of the cultural piece though, is to think about language as different than structured data and for us to all reflect on our own language and how our own language impacts people or how our own language changes people. I mean, it's funny being on a call with some of the British people. This is quote, unquote, the same language. They could still talk podcast to each other, right? There's terminology one of my guys used to term in train. And everybody's like, in train. Wait, where's there a train? And they thought he was talking about a train as he meant in progress as this example. And so that's where some of those cultural things are not necessarily the obvious cultural things because of what's coming out from these chatbots. Well, maybe it is geared to American English, but we as Americans wouldn't think about that. So I would think some of these cultural things we are all going to have to figure it out together. And one of the big things that we've actually, I know, been looking at even from that cultural perspective is we operate in 200 countries and as a result, English language chatbots not going to work in 200 countries. How do we get to languages in multiple countries? And how do we get to a technical language that is used in a payments industry or financial services industry versus consumer language? And those are two different things. So I would say when you say culture, I agree. I just think every company where you're situated are going to face some different cultural issues. Issues. If you're a company that's run fast to break things, your culture is going to be completely different. Your risk tolerance is going to be completely different than where we are in the financial services company where our brand is built on trust, you're going to be very different if you're just based in the US than you are if you're just based in France. Very different cultures, very different perspectives. You're going to have to take to it. So I would say we have our journey, our experience, happy to share. But I do think anybody else is going to have to take and internalize that and make that work for their own company.

Yeah, completely agree. And maybe when you think about here, how do you make the use of AI fit the paradigm of culture of a company like MasterCard? How do you empower people to use AI in a responsible way that kind of drives value for the organization for their own workflows, but also make sure that you keep that notion of trust central while scaling the use of AI.

What we did, as Ed mentioned, is we allowed employees to have access to LLMs and said you can use it without using confidential information. We then deployed internally a tool so they can use LLMs and use confidential information. And so we are empowering our people. We're going to keep adding in functionality and one of the conversations, okay, what are the platforms you're deploying to enable your people? All of that is for allowing experimentation and we are trying to enable as much experimentation as possible. What we then have is we built out an intake process to start getting all the ideas in. And so we had a couple hundred ideas in, this is an example. And then we're filtering through where's the value? Where are the redundancies we can see out of these hundred ten are actually the same exact thing. Not surprising, right? And so when we get to those 10, we say, okay, who can work on them within our standardized, which we call studio product development process? That starts looking at questions such as, do we have the data? Where are the challenges? Where's the market demand? What's the level of effort? There are a standard list of questions that we go through and then also standard stages because we first do some market research before spending time and energy on it. We also would do some pilots do things piloting, internal pilot externally and we have, I think about five stage gates. Doesn't really matter the actual steps, but it is that you should have some formalized process that gets you through to understand what should you actually be spending the time and money and resources on in order to actually get it to a full product standpoint. That product life cycle includes privacy by design, includes security by design, includes AI governance. So those are pre existing processes that we have built into product development. So we just deploy those as an ordinary course once it's in our product development life cycle. So again, we are fortunate that we already have existing processes and we were just trying to figure out how do I take from experimentation down to a funnel because it would be too much to take 100 things and put it into a full product lifecycle management process. So we just had that upfront piece of okay, where are we focusing? So we just built that out and are deploying it more recently to make it easier for all of us to track and get ideas of. But my bet is most companies, or a lot of companies do have a formalized product development lifecycle processes and they should leverage. That would be my suggestion. Let the thousand flower bloom, but go figure out how you winnow down to which ones you actually turn into the bouquet.

You know, we're talking about people here and like, how do you make sure that people feel empowered as a kind of a pillar? Another pillar that you mentioned is AI governance. Right. I'm going to switch gears a bit and talk about AI governance. You come from a pretty heavy privacy and legal background, which I think not many CDOs know much about the regulatory and governance dimensions succeeding with AI and data as much as you do. You already mentioned what are some of the pillars of a strong AI governance program that you have? MasterCard from? Explainability, interpretability, so on and so forth. But maybe how did that background help you define the roadmap for success when it comes to Genai and Data Science at MasterCard? And how does that inform your day to day decision making when you think about the roadmap?

I think it gave me a lot of credibility in the conversation when I said we have to let everybody experiment because I was actually a strong advocate where there were countervailing voices you would imagine about, whoa, whoa, wait, we're going to let people have access to these chatbots? Because I explained where we can build in the controls, how we implement controls to allow for that experimentation in that responsible way, and then also getting people comfortable as we move to deployment and commercialization where we can build in the controls. All of that to be said is there's certain risk tolerance. And part of my job is managing that risk tolerance for the organization on things like that. And so part of that is I question the risk tolerance at times and how we're implementing the risk tolerance. So Security, as an example, is one of my big partners. So I'm constantly sitting there with them going, okay, where's your framework for security on generative AI, how are we doing? What kind of testing? Where are the tools to do it? What are the tools you need? So I would say my background actually built in that level of credibility and trust. I wasn't going to be the cowboy in the room as opposed to, I think if you had somebody who came across as a cowboy, people would be very resistant to go along because they'd be a little fearful here. There was not that fear factor. Instead it was very much, okay, you're going to do this, but how much does that slow us down if we're going to do the experimentation? And that is always a balance. I don't want to slow down people. But on the other hand, I can't launch a chatbot that was not tested for security purposes. That's just irresponsible on all of our parts. So I could have that as a balanced conversation and people would listen to me. So I actually do think it was a great help having that background, having that level of trust within the organization, having the buy in from my partners who knew they could trust me on that particular level.

Great. And we've partnered with a lot of organizations who don't have access to LLMs due to privacy, risk concerns, so on and so forth. And we always hear from them, you know, from the frontline workers, especially like I, I want to convince my boss to get, to get access. What are some of the arguments that you've used in the conversation there to be able to make sure that you're able to create a culture of experimentation within MasterCard?

I would say so first of all, we can start with the privacy. One is we already have standards for data minimization, that personal data is never used in any process unless explicitly necessary. We use indirect identifiers, we do aggregation, we do a lot of things like that. So, okay, people have to just follow that which they've already learned of data minimization. We're not changing anything. We just have to reinforce that we're already doing. So that's sort of one of those easier conversations. The other piece is, as I said, we have a product development process already in place. We're going to rely on it, we're going to double down on it. We're just going to figure out what we actually have to do to strengthen it and where the weaknesses are. So, for example, like I was saying, the testing of a chatbot, at the end of the day, well, okay, how are we testing it? Is it just QA pairs? No, we're going to have a secondary model Monitoring the outputs to see if there's a problem. Okay, fine, Good. So it is those kinds of conversations. Okay, what's your concern? Let's get the concerns out on the table. And it is, how do I address each one of your concerns? Because you can probably go through that ordinary list of security, privacy, confidentiality. Those are your topic. Regulatory. Those are the top ones that come to mind. And you can address each one of them 100%. At 100% level. No, but I would tell you with existing analytics, you couldn't address that at 100% level either. If you walk out the door every day, you are assuming a risk, right? So just living life, but you're going to live life. And so in order to run a business, you're always accepting some level of risk. Where is that level of risk? What's the risk tolerance? What's the risk you're trying to address? Here are the controls against the risk. I think those are healthy debates, healthy conversations. Because different companies, different cultures will have a different point of view as to where that risk culture is and that line of risk is, and different levels of sophistication of how to manage the risk. Those are good conversations. Go have them.

You mentioned something at the end here, like one of the components is regulations. I'm sure you're following the state of AI regulations today. You know, we have the AI act in Europe, we have many state regulations popping up as well in the United States. How do you view the regulatory landscape today when it comes to AI? What should be strengthened and what should be relaxed in your point of view?

So I think we are at a very early stage relative to AI regulation. I think the regulators are figuring out what is needed, what is appropriate, what can they leverage from other places. So for example, a lot like when they're concerned about discrimination from AI, a lot of times they realize, oh, we already have anti discrimination laws, why do I need another law? So I would say the regulators are very much at that learning stage and they're also learning what's the risk with AI. But how do I not shut off the innovation spigot? Because I actually do think in most cases the regulators are actually concerned about that innovation spigot because they understand how it can help AI, can help their populations, help their economy. They don't want to be our economic backwater. So I would say there actually is an interesting and very useful dialogue on that regulatory front. From our perspective, we believe that any regulations has to be principle based, which actually works very well because we have principles. Please, you know, take off from our principles, but it actually allows for innovation because the principles can change over time. We think any regulation should be risk based. Where do the potential harms you're really worried about? Is it physical harms? That's one question. Is it some kind of psychological harm? What is the question there? Let's get some of those harms defined and make the regulation around that. I would say that the self driving cars is quite different than my planning a vacation. Two very different kinds of risks. Let's make sure the regulations fit for purpose. And then the next piece is where are the standards? So Europe can say we need transparency, but what is transparency in the context of a chatbot to be determined? Is it just to tell somebody you're engaging with a chatbot? These are the conversations that we need to have. How do you define discrimination for those of us who are used to the credit space, we are very much used to a standardized way of looking at discrimination. There are like two or three kinds of tests and there's a lot of math behind it that you can get to. I don't think that works with the English language. Right. As we were discussing about how the language can work. So what is an anti discrimination test with regard to a champop, how do you prevent harms like that with regard to a chatbot, where are the standards that you could deploy for that? We are all, I say this, including academia, working on this. We partner with academia, we partner with thought leaders to try to develop those standards because honestly, it'll allow us to innovate that much faster if we know where the guardrails are. But it will also get to better consumer adoption because honestly, you wouldn't get on a plane if you didn't think the FAA had standards for the safety and soundness of that airplane. Right. As we've seen over the last year or two, of course. And so we want consumers to have that full faith and confidence in the AI that they are being exposed to and they themselves are using. And how you build that trust is some of the standards help get to that level trust. So I would say those are ideas relative to where that regulatory landscape is. And the interesting thing, I mean I would come back to the regulators are aware of it. Colorado, when they did what they did, they said we likely have to come back and revisit this. We know we're just putting a stake in the ground, but this is going to probably have to evolve and change. So it was interesting. I'm not used to legislators thinking like that and talking out Loud like that and acknowledging that like that. And it's actually a great dialogue for us to have as organizations, probably many stakeholders to have with the regulators in those kinds of conversations about, well, what does good look like?

I mean you mentioned early in the episode how the rate of change is going on so fast. We're having now much stronger reasoning models, much stronger models that can do math and coding if that slope of progress continues to evolve for the next two years. Right. That puts a wrench in a lot of existing regulations today about what it means to be to have an intelligent system, for example, given current capabilities. Maybe as we end Andrew, I'd love to ask you a few rapid fire questions if you don't mind, on how you think about the future of the AI industry. Right. I'd love to get some quick thoughts of you. Maybe first question 10 years from now, how will Genai stack up compared to other technological revolutions in the financial services industry? Whether Internet, crypto, cloud. Love to see what you think here.

Crypto and cloud I think are examples of things that most people, they're not engaging with.

Right.

It is on the back end side in many ways or very specialized. I actually view it much more like where we are with smartphones, smart devices, that these are going to be things that we're going to engage with on a daily basis that will transform how we engage with each other, how we engage with information, how we consume entertainment, how we create entertainment. So I actually think this will become much more important, embedded in each and every one of our daily lives and change how we live our daily lives.

Okay, second question. You mentioned that AI has been democratized. Right. And in many ways AI is a democratizing force. So good example. I code now much more often now that I have a chatbot that helps me code. Right. I analyze data much more now that I have a chatbot that helps me analyze data and it will change how we work. So maybe over the long term, how do you see the future of work evolving and what will be the main skill that differentiates anyone in the job market?

I think each of our jobs hopefully are going to get easier and more interesting. So if I don't have to write up and rethink how I'm framing an email, but I can give it a couple of bullet points and it creates an email for me and saves a half hour of rebulleting and structuring an email, thank you. It'll just make my job that much easier on the skill sets. Therefore, like I was saying, creativity is going to be that much more important. Baseline critical thinking is that much more important. I just had somebody who joined my data governance team and I was talking to her yesterday and she said, well, should I feel out of sorts that basically I was an English lit, my major in college. I said no, do you have critical thinking skills? And that is going to be what I'm going to ask you to do, like think through the problem, come up with a solution for the problem. I think as things get democratized a lot of these technical skills are going to become less important. To your point, you don't need to learn to write R and Python anymore. You can write one and it'll translate for you and you're going to get to the point you don't need to learn how to write any software code. So some of these technical pieces of knowledge are no longer going to be required and therefore the basics, critical thinking.

Okay, great. And then third question, what is a future that excites you about generative AI and what is a future that scares you?

As you could hear the excitement are things like, just simple things like that shopping cart, right? It's going to brainstorm and make my life more fun is what I expect, more interesting. It'll make entertainment not something that I can just consume, but something I interact with more. So I actually am looking forward to very much that side of it. I think on the scare part we're all scared about the, you know, Skynet. I think that that is always a bit of the fear about can somebody use technology in a nefarious way. I would say that's already the case. We already have debates about how technology is used today. Facial recognition probably a great example of something that people are concerned about as a too intrusive. I think we're going to continue to have those debates. My concern gets ameliorated as long as we are able to have those as open and rational debates about what a society should adopt and what it should control. That's great. My fear would really only arise if it's all hidden behind the scenes and it's not an open debate for society.

Okay, and then final question. What are three books that helped shape your career and journey so far?

So the weird thing on this, so on this one I had to give it a little bit of thought and I would just say a, I'm a sci fi geek, always had been and I really don't like reading those management information books kind of things. Those really just aren't who I am. I find them very short sighted in the sense of A span of history and a span of time. And so I would just say one of my favorite books growing up and I like watching the movie. I always feel like, yeah, it still doesn't do the book justice. Is something like Dune where it's about power structures, it's about life cycles of history over a thousand year period, it's about genetics, it's about limited resources, it's about grand topics.

Yeah, incredible set of. Incredible set of books.

And so you sit there and go, we should all be thinking in grand topics and implementing in our own little way. And that's why a book like that to me excites me, interests me. And I would read it a dozen, I probably read it more than a dozen times as a kid and all the spinoffs from it. But it is one of those things that you could still come back to because it still raises ideas and broader themes for you. So let me just use that as sort of an easy example. I also love history books and so something that a book that again I read a dozen times was the Rise and Fall of the Great Powers by Paul Kennedy, which was very much how countries projected themselves was all dependent upon economic power and how economies grew during the Industrial Revolution and how that was expressed in military power and political power. And so again, grand themes of history I actually think give you a broader perspective than just worrying about who moved my cheese. Honestly for me personally, because I'd rather think where to your point, where are we going to be in 10 years? That's an interesting question. How do I get to where I want to be in 10 years to that vision of that interactive entertainment? Just as an example, what are the building blocks? Where is a larger societal piece that gets us there? So I would just say offer those up of examples of books that got me excited. I know other people have different points of view.

Okay, perfect. And finally Andrew, before we wrap up any final call to action before wrap.

Up today's episode, keep an open mind at all times. Just listen, learn, absorb. In my career I've changed, as you probably saw on my resume, change jobs, roles, what I do multiple times. It keeps life interesting. Generative AI will keep our lives interesting. Just be open to to it.

Okay? Amazing. Andrew, thank you so much for coming on Data Framed.

Thank you for having me.