Fireside Chat: Engineering Leadership in the AI Age

I'm super excited to introduce you all to Andrew Shea, CTO of FanDuel, an amazing engineering leader going through the changes within his own organization. Andrew has tons of experience building and leading R&D teams with some incredible thinking on AI, how we embrace it, how we integrate it, and where it's going to take us all. And with that, welcome Andrew.

Hey everyone. Thanks for having me, Andrew. I appreciate it.

This is all the Andrews today. Everyone gets the, the full thing, right? Well, so today, as you, as you know, we're talking a lot about AI stuff cuz it's so transformative and so on the nose today. But to help make sure everybody knows where you're coming from, if you're willing, can you introduce yourself, your company? You know, where you're coming from. Um, tell us what your world looks like so we can actually understand your reactions to all this AI stuff and contextualize it.

Yeah, certainly. So my name is Andrew Shea. I'm the CTO of FanDuel. And for folks who may not know what FanDuel is, FanDuel's, uh, America's number one sportsbook. So we actually do provide an online sports and gaming experience for folks that want to bet on sports experiences. Uh, I've been at FanDuel for about 3 years and manage all of the engineering and software development there. And prior to FanDuel, I worked for software companies like Compass and Palantir, and I really had a lot of, you know, experience prior to this with AI and machine learning, even before GenAI was a thing, you know, the last couple years that it has really buzzed around.

Yeah, and I think we were chatting the other day, you gave me a little bit of background around the wide diversity of kinds of companies you've had. So like, there's the job you do today, but like, you kind of gave me a longer history that kind of different kinds of companies, right? I think it might be useful for folks to hear a little bit about that, right? Just because I think we're at a moment of change, and I think if you've been in the game for X amount of time, worked at this kind of company, you look at it a certain way, and if you think the other way, you think it has differences, right? So if you don't mind, a little bit more history. Yeah, 100%.

No problem. So just a quick background. I grew up in Boston, Massachusetts area in the suburbs, went to school in upstate New York for computer science. After school, I actually went to a company called Sandia National Labs, and it really gave me an opportunity to work on a lot of different areas across the government space. And, you know, this will pretty much date me, but in general, like the tech shifts that have happened across the few decades, I was kind of in the new cloud era. So I remember big data and cloud being the themes of, of really Sandia, like how do we take enterprise monolithic stacks into the cloud? And, and then I moved my journey on to the startup world. I joined Palantir Technologies in Palo Alto. It was a long time ago, about 14, 15 years ago when there's roughly like 100 employees. And for us it was really about getting data from the actual enterprise systems of the old Oracle stacks that were in-house to really into the cloud, as well as how do you get it to the hands of users quicker. So Palantir is really at the forefront of taking enterprise data to the analyst desks and moving that. And in my journey, I realized I wanted to continue going through startups and actually went through a real estate tech startup called Remine, which later I actually got the opportunity to join Compass, which is a SoftBank-backed company that went through public IPO a few years ago and really evolved there. And so FanDuel is, for me, has been the newest job that I took on since Compass, and it's been a journey since.

Awesome. Well, I was actually joking that I was able to catch you here today. I think you're in DC today, is that right?

I am. Yeah, I'm always on the road. Right, right, right.

Okay. So this actually leads to actually help me help folks understand kind of FanDuel. Like you told me you're traveling 3 out of the 4 weeks. And so what is the— like, help everybody with scale on like FanDuel. Like how big is it in terms of, you know, engineers and offices and why are you traveling so much?

Yeah. So FanDuel is a pretty big global footprint. So just to start with FanDuel, we're about 4,500 employees. We're actually— we are actually headquartered in New York City, but we have offices in LA, Atlanta, New York, Edinburgh, Scotland. And as part of kind of being part of Flutter, which is our global parent company, we have software development centers in Portugal, Romania, Australia, and a few others. So it really does kind of take me away and have to kind of, you know, globetrot around.

Got it. Okay. Well, that helps set scale for everybody on this. So since the theme today is actually about AI, and I think I'd admit that I think we're all learning about it and how to actually tackle it. It really comes down to trying things, right? I think that's what the theme has been in the last few years. Experimentation, I think, is the way I think about it. Like, at an org that big, how do you experiment specifically on AI? Like, how do you, you know, AI coding tools, like, how do you try them? Like, does everyone do it all at once? Like, how do you— How do you try stuff?

Yeah, this is like the magic question. I think a lot of people in organizations will have different answers. And I think for us—

Yeah, that's what we're going to hear from you.

We have like 2,000 people. So when my CEO is asking me, what are we doing here from a perspective of dipping our toes in there? The first part for me from an engineering perspective was really how do I leverage these tools and get it to my employees' hands faster? So from an engineering perspective, it was very straight out of the bat. How do we give them generative assistance tools where code actually gets generated or helps them in their job? So I took it upon something, as you know, how do I give the developers something that they're already probably doing? So in the— in a couple of years ago, people are still using Stack Overflow, they're Googling code, so they're already getting code assistance in one way. So this is kind of like, how do I integrate it in their IDE?

But do you use it first or did a team member bring it to you or was it like a corporate direction?

Yeah, we picked a few teams. You're right. We picked a few teams actually, because I think one is we wanted to make sure You know, when you pick a tool, everyone's gonna grumble. So we wanted to understand how do we experiment it and get a few people to do it. So we tried it with a few development teams to get some metrics first. Yeah.

So like, okay, so you, you kind of made a, I, I'm guessing that people kind of were picking it up organically here and there, but like you said, hey, we're gonna do the experiment. We're gonna pick some few teams and try it this way. Like, it is that curiosity. Yeah. Because, because I don't get, like, I don't get to touch the code and now I get to touch the code again with this kind of new set of toolset. Did this get your hands back in the keyboard to try it yourself? Or like, you know, like, are you part of pilots?

I do on-hand keyboard a little bit. In general, that's been my philosophy. But I think the biggest part is when we were experimenting with— part of it was asking, we had to ask our developers, one, do you find that it's actually helping you? Number one. And then two was like, how do we measure this thing? Like, how do we actually understand if there's an uplift? So that was kind of part of the, how do we go from experimentation in some of the teams to globally, I give it to all my developers have access to. So that was kind of the steps. But yeah, it got me back to the keyboard a little bit more because I wanted to understand what they were experiencing.

So it sounds like you guys did some isolated experiments where, you know, one of the criteria coming out of them is asking them, do the engineers feel like it's actually working? It sounds like you guys set some success metrics yourselves too around that. Did you do that a priori or did you say, like, did you kind of determine the success metrics as you were going along?

We set some general, uh, actual metrics. We, like, we didn't want to prescribe anything like what percentage of lift or how many more minutes of productivity you get. We wanted to actually feel like, you know, it's a side-by-side companion. So for us, it was like, is it something that's actually helping you? Is the quality of the code that's generated for you actually creating uplift versus is it accurate and you're having to, like, spend time reading and correct it? So We evolved, I would say, our metrics in the beginning to understand, is it actually useful? Are we using the right tools?

And like in this kind of pilot trial period, in that, this sounds like it was a positive experiment. How long was that, you think?

It was about a month and a half. We wanted to sprint as fast as possible to make a determination to jump in.

And is this a formalized thing you guys do with all new things? Like, have you had things that follow this and don't work?

And like, oh yeah, yeah. I mean, that's, that's the nature of FanDuel. Like, we actually like to fail fast. And so I think for us, 30 to 45 days experimenting was already kind of slow. But I think by the nature of, you know, introducing something like generative AI into our code, we also want to understand the security aspect of it. Is it now looking at my codebase and GitHub?

Yeah.

So I think as part of that analysis, there was parallel work at the governance level of just making sure that the tools we just provided them with actually has security and, you know, guidelines for protecting what we built.

Got it. Once you get past that point though, like, "Hey, I think this is a thing we want to do," how do you drive change across the organization? I'll ask this in two ways. One, because we were just talking earlier about how a lot of organizations aren't getting the usage they want on some of it. Some of this is change management. The pilot might go well, but some of the folks that are rolled in don't get it. But also the other side of it too is like, frankly, it's a time of a lot of change in the world as a whole. So how do you support people that are like, I don't wanna do this, or like, is this gonna, like, how do you manage through that stuff? Right?

Yeah. I, I think any leader in technology, they have to apply the equal change management to the amount of efforts to be clear. So I think you bring up a very good point there. The first part here is like, for us, we actually did, leverage like GitHub Copilot as the code assistance tool, we leverage GitHub to also train our employees on what the usage can look like. So instead of just saying, hey, we found a tool, it looks like it works for a group of people, we're going to just deploy it, we actually embodied them with the here's best practices, here's examples how you do that. And then we actually gave them opportunities like the hackathons and like small 0-to-1 efforts where we say we want you to only use this to understand its limits, for instance. So I think we gave them some guidance and freedom, but also here's some best practices. So we didn't want them just dabbling and figuring out, but I think that's one way to do it. It's not necessarily the right way, but I think at a scale of 2,000 engineers at FanDuel, you do need to give them kind of the, here's how, uh, companies that develop the tools are leveraging internally as software engineers. So yeah.

Well, okay. So you, by the way, earlier you told me you piloted with a couple teams, but let's say, you know, you've got 50 people using it and now we're trying to move to 2,000. Do you deploy all 1,950 people all at once, or do you kind of move geography by geography, or how do you step into a front-end teams, back-end teams? Like, how do you do it?

Yeah, so our actual— when we experimented, we did see where there were opportunities that it worked very well in some areas where there's some challenges. And so part of it, from like, you know, a tech perspective, you know, the recognition of certain programming languages were stronger than others. So in certain teams, if, you know, we actually don't have consistent programming language framework for all our products at FanDuel. So some teams would benefit instantly from things we saw, areas where you have heavy QA and like, how do I actually integrate things like generation test scripts? Those needed to be experimented a little bit longer. So the way we deployed it was taking it into the teams of development where we knew there was more confidence that the code was quality, was a little bit better and more consistent. And then with the other ones, we kind of gave them access to it with a disclaimer of, hey, recognized some challenges you guys should continue to evaluate.

Did you have any teams that rejected it? Whole Hog?

I, I wouldn't say they rejected it. I think some teams would, were slower in terms of getting a higher percentage of people using it, right? And so when I say using it, it's kind of like we looked at metrics of how much percent of code actually was being generated by the generative assistant, for instance. So when you saw lower numbers, it doesn't necessarily didn't note that the group was resisting it, or maybe it just didn't work as well. But we did kind of work with teams, say like, you know, what is going on? Do you feel like this is useful? And we didn't force it. We wanted people to embrace that we're here to help as a resource versus this is replacing what you do or move faster kind of initiative.

Got it. So I'm hearing some goals, some macro goals, some support structure, and some telemetry along the way too to make sure that you're getting those yields out of that stuff. I'm assuming, though, that change isn't done. That's been my view on this. It's just like, hey, that was a good step, but there's going to be a whole new paradigm and set of tools coming around the corner too, and you have to do this all over again, and it's a different change on this stuff. My reaction is it's moving so darn fast. I'm deeply empathetic of the size of organization because in some ways technology is moving so fast. but the orgs can only change so fast too. And so there's like a balance across those two things. I think there's like, that is actually potentially the industry, uh, velocity limiter, right? Which is like, it, you know, I'm not sure the orgs can take the change as fast as it's coming on that stuff, right?

Yeah, I think it takes a balanced approach, to be honest, and it differs by organization. And so for something like Fandul that has a lot of products in regulated space, we still have to deliver against a very, you know, tightly coupled industry. And so I think in some spaces you take the pace that is critical for your organization. And I think it's really for people to say there's so many tools and so you can also get distracted, right? So every month you're pivoting or experimenting. I think employees get frustrated.

They want to know what is it that you're doing. Because there's all this stuff coming. So do you have a cadence with which, like, is there a reevaluation cadence or like, because you I mean, I'm assuming at the rate of these things, you can't try every tool that comes across. It just like, it seems impossible, right? Like, um, yeah, yeah, yeah. But if you sit there for 2 years, it's probably not a good move either.

Yeah.

So like, is there ways that you kind of rhythm into it?

Yeah. I mean, things that you're trying to, like, I would say stretch across the organization, we try to do that and evaluate within a 3 to 6 month time period to say like, is there something we should experiment that reaches globally all of our teams? But I think culturally what we want to do is create some autonomy where some people actually find tools that are working for themselves or the organization. And then locally, how do we support that? Meaning like if they want to experiment and get some licenses, we're not against that as long as obviously it supports governance and security, but we want that. So I think there is some organic people self-discovery where teams want to find something that works for them. And it's up to us as organization leaders to figure out how do we assimilate them in local spots versus force everybody to Pick one path.

Got it. Well, hey, so implicit in this, all these things, how do you think about outcomes? So I think you've kind of nosed into this a little bit, but like, you know, look, we're all doing all these AI things and, you know, frankly, I think there's a lot of variance around what kind of things actually come out of this. You talked a little bit and like, okay, so you got lots of tools, you got a lot of variance, you got CFOs going like, hey, all this money How much is being spent? Like, what's the yield, right? I love you CFOs, but it is, like, how do you think about that, manage that for the organization, right?

Yeah, I mean, like, look, obviously I also have a CFO, so sorry, David, when I say this, but I think the key here is this, like, you know, at the engineering leadership perspective, one is especially when it's new and employees are embracing it, their first doubt is, am I going to be replaced by AI if I stop writing code? Or if I have code that's getting generated, am I basically writing myself out of a job? And so like, there's a little bit of skepticism in the beginning of that. So I think the, the part about it is really getting the teams culturally past the safety point of like, look, we're here to actually figure out ways where you actually get your job done and spend time on more exciting, challenging things. So examples of this is like, you know, even for QA or developers, they have automated test coverage now. Like they have ways to make sure like, hey, when you build code, there'll be things that help you do things that you don't really want to do maybe, or document your code. And so instead of us pressing you on where's your documentation or code, there's things that give you that uplift. So I think really giving the, the, the data to people saying here's how much time you don't have to spend on these things to go do things like new initiatives, they kind of value that and they realize it's an, it's a plus versus a, it's, or, you know, kind of and/or kind of, uh, paradigm for them.

So is that in your planning processes now though? Like, you know, are you working with the business, your CFOs being like, hey, We do these things and I squint a year out, it's going to be like this kind of load. Or I don't know, like, how are you guys thinking about that? Or is it too early in the curve?

Yeah, no, I look, look, there's, there's some things within the engineering org, like what I mentioned, the productivity and code gen. That's one area. But I think when you look at AI in general, especially where we are, a lot of use cases have been very successful in businesses is at the enterprise productivity and automation perspective. Right. And so when you are working with the CFO or when you're a technology leader, There's obviously, you look at your organization, say, look, the critical part is, are there areas that can create efficiencies? Not necessarily with your, your, your staff or human resources, really like how can we get done things well, more accurate, less human intervention and things like that. So we do set goals around areas that we look at functions or specific initiatives where it's a, requires a lot of people. An example in finance, reforecasting of data, or every time we do budgets every quarter, you're already preparing for the next cycle. So that, those are examples of types of goals we set.

Got it. Yeah, I think it's, uh, it'll be interesting. Um, I mean, I, like, my view on this is like, we're, we're all at a very dynamic stage on this stuff, right? Like, I, you know, my two cents is that it'll be different 5 years from now. I don't know how it's gonna be different, but it's gonna be different on this thing, right?

Uh, I'm actually kind of nervous. Like, honestly, like, I, I always thought my, I have two boys, by the way, I have an 8-year-old, 4-year-old. Uh, and I always thought, you know, I, I go, maybe my kids will go into computer science one day and be like me. I realize, there might not be the same coding. They might just be like, pump engineered, or let me go tell this robot what to do. I don't even know if they're gonna have to drive cars, to be honest, by the time they get their licenses. So it will shift.

I mean, I heard that the new robot cars are gonna open in DC shortly, so you guys are gonna get them soon. Might, yeah. Yeah, yeah. I don't know, like, it's— that's such a good test. Like, what would you recommend your kids go do, I think, is a, is a good question.

Well, I told them pick up a trade.

I know. Plumbing's pretty good. I'm good at that. I can solder pipes.

Yeah, exactly.

I'm down with that stuff. Real hard skills. So when you look forward though, like, what are you thinking about? Like, are you, you know, both in terms of technology, your team, the company, like, what are you excited about? What are you worried about? Like, what are you watching for? Like, and pick any horizon you want, but like, not today, but like in the future. Like, what are you thinking?

Yeah, like, look, I'll go across the spectrum. In terms of worry, you worry that you're not doing enough, right? You worry that you look left at the organizations that are similar to you in size, or just in general, you're like, they're moving fast, they're adopting quicker, they're, they're— screw it, we're going full in. Like, it— and we may be wrong. And so they're the organizations that are very aggressive in nature and in that strategy, and that's nothing wrong with that. And then you look to the right, say, well, I'm actually doing better than some people. So then you get complacent. So I think the worry is there's no good meterstick of understanding how well are you really doing. So I think right now the way we are doing it is, is it helping our business to what we think it could do? Right. And so I think in general you could always be more aggressive. So I think that's kind of how I would bubble that.

So, so yeah, so, so is your like paranoia around am I getting the full utility of the opportunity today kind of thing? Right.

Yeah. I mean, everyone is having that. I think, you know, all our executives are giving us pressure in that regard and for us. It's kind of like, it's, you know, how do we respond to a CFO or a CEO saying, can we be doing more? Can we go faster? Are we leveraging AI deep enough? I think we all answer that and there's no meter stick to say what is good or what is fast.

Yeah, it's a, well, it's an interesting question around like how to actually think about the business in the context of the opportunity here, right? Yeah. Well, okay, more fun question. Now that we're going to get this, like, not that you have a ton of time, but when you get a moment here, do you got any side projects for you that you get to like test out some of this stuff? Like, if you were to get your kids in here and actually play around with these tools, like what would you be, what fun things would you get to go build?

It's funny because, you know, my, my, one of my oldest kids has gone through like kind of coding classes before and like, it's funny because I'm a computer science, it actually takes me a quick step to understand how to do that, right? So, even at the lower levels for kids coming in, they're coming in advance. So I think you'll just see a lot of that. So, you know, on my free time, I will try to help my kids around the house and pick up things, but I have to learn everything. Even their math homework is not the same anymore. So I think everything is changing quickly where we are still learners ourselves. So it still feels like work even though it's fun.

Awesome. Well, so speaking of the future and the next gen, Do you have any quick tips here for other folks that like want to follow in your footsteps, future leaders, the next Andrew Shays?

I think the path that has kind of kept me where I am was follow what makes you really interested. I think where you continue to find something that you're continuously learning, you're surrounded by people you really enjoy, and the mission is going to always lead you to something where you look back and say, that was great. And so every chapter in your career, will lead to something. I worked in real estate, I've helped launch satellites into space, I've worked data in, in national intelligence agencies, and now I'm working in sports betting. So wouldn't have thought I landed here, but genuinely, I think it's, it's a journey. You should enjoy your journey and follow what really makes you happy.

That's awesome, right? I think that advice can be used for everyone here listening to this, but also to your kids too. I think that's a, uh, good place to be, and I'll take that, uh, home for this. Well, with that, hey, Andrew, I really appreciate you being here with us. I think it's such an interesting time and your perspective of the organization really is helpful for all of us. So thank you very much for being here.

No, thank you for having me. I appreciate this opportunity.

Yeah, right on.