Stories from the Illuminate Awards Winners: Celebrating Engineering Excellence | GLOW 2024

Hi everybody. My name is Phil Braden. I'm one of the co-founders here at Jellyfish, and I'd like to welcome you to our panel on celebrating engineering excellence. I'm honored to be joined here today by 3 executives who represent our Jellyfish Illuminate Award winners, and each of whom has done a really impressive job building and managing a high-performing R&D team. The first award is to Keller Williams, who wins our Sea Star Award for their achievement in DevFinOps excellence with Jellyfish and also improving the productivity of their engineering team. Chris, would you like to introduce yourself?

Absolutely. Thank you. Hey guys, Chris Cox here. I'm the Chief Technology and Digital Officer for Keller Williams, which is an international real estate firm. And in my role, I have responsibility for all of our engineering and development as well as product management. Design, data and advanced analytics, and our innovation and corporate development platforms as well. Really pleased to be here.

Awesome. Thank you again, Chris. And our next Illuminate Award winner goes to Genius Sports, who wins our Big Wave Award for their achievement in driving transparency and alignment across their organization. Michael, would you like to introduce yourself?

Sure. Thanks, Phil. Hi, so my name is Mike Patel. I am the Chief Technology Officer at Genius Sports. I joined the group in 2011, and I feel incredibly privileged to work with such a talented team that's built an incredibly successful business over the last 13 years. At Genius Sports, kind of what we do is we work with some of the biggest sporting leagues in the world, like the Premier League and the NFL, but also with large sporting federations like FIBA, who are the International Basketball Federation. Our vision is kind of to redefine experiences through technology. But I often contextualize that to our tech team as we want to be the operating system for sport. So that means we work across a diverse range of fields from competition management to coaching tools, data collection, betting, advertising, video streaming, Sport is kind of the central connecting theme for us, and our tech stack is, is equally diverse, right? We work with Rust for some of our lower latency systems, to Kotlin, which we use for our streaming platforms, to C#, which we use in a lot of our betting systems. So we occasionally need a bit of help from an engineering platform to make sure we're keeping everything on track and keeping everything efficient. Nice.

The diversity part came through in some of the conversations we had leading up to this, and I think it's an interesting part of your story. So thank you for that. And as luck would have it, Michael didn't say it, but Genius Sports also happens to work with another one of our Illuminate Award winners, which is FanDuel, and they're winning our Best of the Bloom Award for their achievement in laying a foundation for both team management and cost capitalization best practices as a blueprint for a broader organization. So Shane, would you like to introduce yourself from FanDuel?

Absolutely. Delighted to be here to share some of our story. So my name is Shane Sweeney, SVP of Technology here at FanDuel. FanDuel is the number one sports betting or online sports betting and casino operator in North America. We have been in hypergrowth mode over the last number of years from an engineering perspective, building reasonably complex, highly distributed systems across all of North America and, you know, building out a team, you know, that scales from 3 or 4 years ago that was 300-ish engineers across 2 or 3 dev centers to over 1,500 engineers now has been a journey all by itself, not only dealing with the technical challenges of the business growth, but also the hiring challenges. And again, this has been a great platform for— or Jellyfish has been a great platform to help us really dig in and understand what's going on across our engineering world. I come from a background within the Flutter Group, which is the parent organization for for FanDuel. So after 16 years of building out platforms and building out teams, you know, it's been the last number of years have definitely been some of the more challenging and most interesting parts of what I've been dealing with.

Awesome.

Thank you, Shane. So I'm psyched to have all of you here. You've all worked with Jellyfish for a little while now, some of you going back years. I'm thinking, Chris, you might be the most tenured Jellyfish partner of the three here. And yeah, I think it's been, it's been a wild few years for, for the industry and for each of your companies. So I think there's a lot of interesting perspective that comes out of that. I also think it's, you know, we look at like gaming versus, versus real estate. We're covering a pretty wide spectrum of, of industries. But there's also, I think there's a lot of commonality into what I hear from, from each of you and from some of the things that you've been doing during the time that we've known each other. I think one thing that would help the audience, or one kind of starter question, is how do you quantify success for your engineering teams? Because I think there are some similarities and some differences, and I think everybody is always trying to kind of get that right. Which one of you would like to take a first stab at this one?

I don't mind taking the first stab. All right, sure, you can go for it. Thank you, Phil. When I think about quantifying success for our engineering team, I think about 4 things. I think the first, the first is to what extent do we have aligned autonomy in a diverse team? Are our engineering efforts focused on the right things? And what measurable, visible progress are we making against those efforts? And how do they align to our wider business objectives? It's incredibly important that we're working towards the right goals. And we want our teams to be autonomous and to decide the how, but still have sight of what we're trying to achieve. The second thing I think is how much of our time are we spending creating business value versus support and maintenance, keeping the lights on? How much time are we spending investing in productivity initiatives? Whether that's refactoring systems, adopting new technology, improving developer experience. We want to strike a careful balance for how much time we're spending on our product roadmap versus things that are not business-facing. Third is how many service interruptions are we having? What's the quality of our system look like? If you think of on the left-hand side, you have the tempo of the business, like how fast are we going at delivering new features, right, then you also have the counterbalancing pressure of, you know, is the system stable? Are we delivering features that are of high quality, right? And are we causing downstream issues for our customers? And I think it's important to keep those things balanced and in check. And also to make sure when we're dealing with service interruptions, how do we respond and recover from them? And also how do we, you know, do we repeat mistakes or are we learning from failure? And fourth and finally, I also think as a whole, it's important to know how quickly are we able to take action moving work through our development process. Can we set a rapid pace of development and deliver predictably against key priorities for the business? So I think those are 4 things that encompass kind of my system for quantifying success for the engineering team. But also outside of that, I think a little bit about the qualitative measures, the things that are harder to measure about elements of engineering culture. So one thing I'm quite excited about with the Jellyfish roadmap is DevEx, right? Like surveys that we can use to find hotspots where developers and engineers are struggling to actually make ground and the developer experience, the systems that they're working on is not that good, right? Like we use, we use Picon to so the overall health and well-being of our engineering population. But we are very interested in having something that's targeted and tech-specific to help us improve our tech.

Awesome. That's a long list of things. I'd love to— I think one of my regrets for this whole session is that we've got 30 minutes because I think I could spend an hour and a half with the three of you. And I think the audience would appreciate it too. But thank you, Michael, for outlining all of that. And we'll touch back on some of it. Chris or Shane, would you like to give your slant on how you measure success for your engineering team?

Sure.

I, you know, I think it's very, very similar. At the very highest level, it's about pace and quality, very similar to what Michael outlined. I would add agility. We work in a fully end-to-end agile operating model. So We are very focused on making sure that we have deployed resources deployed against the right work at the right time. And then, you know, and operational stability. So at the highest level, it's those 4 things for us, very similar to Michael. All those, all each of those 4 have metrics that, you know, ladder down. But, but those are our big 4.

When, when you talk about agility, like Michael also, Mike also talked about it, I think, in terms of being nimble. Do you measure it in, because sometimes this is measured at the level of, you know, how fast are the teams responding? So, you know, lead times and cycle times kind of metrics. And then sometimes it's, you know, there's a level up of how quickly do you, you know, reorg or restructure teams to go change missions, for example. Is it some of both of those things or how do you think about it?

It's definitely some of both. I mean, we work in this end-to-end model that is defined—

sprints.

And so we are working on a rolling quarterly basis to, you know, to step back and reprioritize all work that we have in the queue and move resources around accordingly based upon priority. We've moved away from a sort of a project nomenclature or a project way of doing business to a model that we, that we call persistent cross-functional teams, which means we've aligned around key customer experiences and we've persisted or dedicated teams to those experiences. Those, those teams work in roughly 2 to 3-week sprints. And, and then across our organization, we pull up no less frequently than quarterly to restack priorities and to realign resources as we need to.

That's great. So you try to keep the team stable and then focus on the operating mechanism that drives those teams.

Yes, absolutely.

Awesome.

And Shane, last but not least, I know some of your story overlaps with this, but maybe give your spin.

No, absolutely. I got— it really resonates what Michael and Chris said here in the last few minutes. I said it in my introduction, you know, Fanduil has been on this hypergrowth mode, you know, 300 to 1,500 engineers over a very short number of years, a complex environment that we have. We actually want to be able to have all of our teams operating in that agile startup type fashion while maintaining the enterprise levels of visibility and governance that our business and commercial colleagues want to have, providing the best quality and delivery of products out to our customers and ultimately providing the best customer experience to our customers. And that's in all aspects, both from an operational perspective as well as just from a product features perspective. That means we need to really make sure that the right decisions around investments are being made, that we're working on all the right things. We want to be able to enable more innovation and see how we can really reduce the overheads associated with that growth and the complexity as early as we can in the lifecycle. And that ultimately means getting insights down to a team level or an individual level was a key criteria and success criteria, you know. But at the same time, we also didn't want the gathering up of data to become an overhead associated, you know, adding more and more work to our engineers. So actually really having a a platform capability that was able to link back into the systems and tools that the engineers use today to surface up all that meaningful data to be able to enable us to make these decisions. It's a key part to, I think, what we need to continue to do and where I feel we've succeeded over the last number of years, particularly since we've introduced Jellyfish.

Yeah, I feel like for you, the job keeps getting harder, or you have to, you know, you make a step and then you grow enough that you kind of have to do it all over again because the team is that much bigger and the bar has gotten, you know, gotten that much higher. Maybe in line with that, can you share a little bit about how you've improved operational visibility over time, as you know, during this growth and, you know, since you've started working with us?

Yeah, just to give a bit of color. So we have about 22 live online platforms in North America today, and it grows every, every month. When we deal with having—

How many did you have a couple of years ago?

When I, when I joined about, when I started here in the US about 3 and a half years ago, we started with 5. And every single one of those includes the building of a data center, the deployment of infrastructure, the deployment of the platform, the configuration, and each one is distinctly different. That operational overhead was extremely large.

Just doing the math, at least one of these per quarter.

Yeah.

And then some. It's seasonal based on, you know, typically we launch just prior to some of our biggest periods of time. You know, states will want to launch before the Super Bowl, for example, or before the NFL season starts. And actually that alone as well is another key challenge because just at the point at which you need to be the most prepared, you're changing the landscape. And again, over the last number of years, you know, that operational overhead and with team growth, if we didn't have the level of visibility that we have today, we wouldn't be able to really understand and make better choices around how to optimize ourselves and how to really unlock the full potential of what is the giant organization, the engineering organization that we've built up. People, I think the teams would be not focused enough on in certain areas. And ultimately, getting that visibility has been key to be able to really hone in on decisions that we might have made in the past that actually made a lot of sense in one mode of operation, but moving to the next phase of maturity needed us to pivot.

Nice, nice.

A thing that is, that's relevant for all three of you, but maybe is newest, I think, for— or at least this is like the most recent year's work with Keller Williams. Chris, you're leveraging Jellyfish as part of your relationship with your finance team. Can you talk about how that has changed how you operate or has impacted the way you run your organization?

Absolutely. So So I'll start with, you know, what was the essence of our, our challenge to begin with? And, and, and it was a, in a very inexact and precise way to look at our, our software capitalization. And for those financial familiar with the financial ins and outs of software capitalization that, you know, that ends up directly impacting GAAP-based net income, depending upon how you approach it. And so one of the things that we've really sort of dove headfirst into with Jellyfish is tuning our environment using Jellyfish as a critical and key backbone component to improve the precision with which we log engineering time And, and then downstream, how we can, how we can make decisions around CapEx versus OpEx. This has been really, really, really critical with our, our financial team, our CFO office, because in the past we had pulled up every month and done what I guess charitably could be described as a swag around the, you know, the proportion of of engineering time allocated to capitalizable events versus non-capitalizable events. So in a very interesting way, we've, you know, we've integrated Jellyfish into our broader infrastructure with Jira, GitHub, and a number of other tools to create this radically different and much more precise way of looking at where our engineers are spending our time. And the acute nature of the way that we do that allows us to make, you know, to make decisions both on a month-to-month basis from a financial standpoint as well as a longer-term financial forecasting standpoint. And that has yielded real dividends for us. So It has all the benefits of, you know, of us understanding how to manage a large engineering resource base, but it has the added benefit of allowing us to run what I guess we refer to as a DevFinOps shop that where we're in a much better space than we were just, you know, 24 months ago.

Yeah, that's great to hear. And it's been great to see. It was really nice meeting your team a few weeks ago and hearing how I think one of your team was referring to it as this pleasant surprise, like you'd been heads down using Jellyfish on the operational management side, and then realizing that you can leverage it for on the DevFinOps side as well. Shane, I know this has also been very relevant for Fanduil and Flutter. Is there anything that you'd like to add? I think your interesting part is that your parent company has gone through an IPO and like that was also a driver for increased rigor. Swag's become less tolerable when people are staring at them during a financial event like that. How has, anything to add or any compare and contrast to what Chris just talked about?

Yeah, so our journey, we've had a situation in Fanjul where The teams were all focused on building new products. Our finance teams were trying to manage that R&D credit and the DevEx side of the accounting and the books.

It was easier when everything was new.

Yeah, exactly.

It's not a hard problem.

Yeah. And when you're scaling and growing at the rate that we were, you know, even subtle changes in percentages are very meaningful. Going from 300 engineers at 80% capitalization rate to 1,500 engineers when it's a very different percentage rate, that can, without having that forward level visibility, that was a struggle. That was the one, the primary reason why we looked at this tool is really to simplify that. Again, we didn't want to create overheads on the team. The moment that actually we moved away from that spreadsheet model, which was very error-prone, we actually realized that the treasure trove of data that we had was very meaningful, our finance organization, just as much as it was on the engineering side. And they brought that up to our parent company. And two things were happening. One, everybody in the entire group of Flutter was experiencing scaling opportunities. This was becoming an increasing problem. You know, there's over 8,000 engineers in the entire Flutter group worldwide across 30, 40+ countries. Coordinating this at a group level for even where we were doing our reporting in the UK, it was becoming more of a challenge. The moment that we shifted over to the New York Stock Exchange, the rules of the game very much changed. And, you know, the SEC has very specific guidelines around how these things have to be managed. And actually what we did was we were able to leverage the work that we did, which is inside of FanDuel, and educated the rest of Flutter. So actually, nearly all parts of the Flutter organization have built upon that capability because they saw the value. Again, it's that the richness of this data is hugely beneficial without having that cognitive load on our engineering managers and the error-prone results that, you know, were happening every now and again that required controls put in place. And again, it just simplified everything to do with this space.

Makes sense. Thank you. Thank you so much, Mike. I know, I know that when we were talking, you mentioned, you know, how Jellyfish has been relevant to discussions with your CFO. But you've also, I think part of what we really appreciate about what Genius Sports is doing is how you've taken some of like the board slide templates that we've produced and some of like the capabilities that we're giving to you and packaging them to drive executive alignment, not just with finance, but across executive stakeholders. Can you share more about how that's working for you at Genius Sports?

Yeah, sure.

I guess it comes back to a little bit of what I was saying about it's a very diverse org and we work across a lot of different investment categories, like I'll call them. So we might be working on competition management, some pricing stuff, some risk stuff, anything from officiating to video streaming to advertising, right? The, the hard thing for us, I think, especially with our senior management team, is keeping track of investments, keeping track of where we're spending our time and energy.

Because of the diversity.

Correct. Yeah. And also coupled with the fact that we operate a mature system that requires modernization, right? So sometimes it can be a bit unclear like where engineering resources are being allocated. So we, uh, our IS&T report, um, which is something we send out every 6 weeks, uh, it reuses actually parts of the 5-board slides template that, um, Jellyfish offers as a, as a guideline to summarize basically how we're doing as an engineering team. So it covers what deliverables we've completed, like not just a concise summary of what business impact they will have, right? But also the investment cost. So we sync a lot of our salary data, median salary data split by location, and that helps us actually keep track of the amount that we've invested in initiatives over time. And that rolls up into our SMT reports.

So you can have the conversation at a business level rather than what are all the—

That's the goal, right? Like it's very Engineers are very focused on everything they're doing and all of the detail, right? And at the S&T level, we want to create something that's easy to digest and it's actionable for other members of the executive team and the senior management team. So it also includes how much time are we spending on roadmap work, on support and maintenance, how much time are we spending keeping the lights on, and also on productivity work, so the modernization efforts that we're spending our energy on to simplify code bases or adopt new technology, things like that, so we can help set the right balance going forward. Third, the quality of our systems, right? Like how many service interruptions are we having, right? The impact of those service interruptions, right? It's important to keep that paired with new feature development so the management team can, can keep track of the broad stability measures of the system, the security of the system. Like if you have a lot of different software systems that are in play at the same time, we have a great internal pen testing team that find creative ways to break into our systems. And we also surface those measures, like how are we doing from a security perspective? And we also cross-validate that with an external red team. Who we use to make sure our systems are robust and will repel attackers. Finally, to close out our SMT report, we, we also include measures of hiring and staff satisfaction so we can keep a holistic view of how the engineering team is growing and how the engineering population feels about strategy to well-being to like the systems that they work on and it overall gives the team a concise kind of clear pulse on how the engineering team is performing at a fairly regular cadence. So it's been a very helpful tool to use as a foundation for that work.

Yeah, and it sounds like, you know, part of the value there is the consistency, right? Like you're sharing this on an ongoing basis. I assume when I hear you talk about all the different things that you've got to take on, you know, there's probably a discussion or three about a thing that you can't take on. And so, you know, making sure that everyone's on the same page about what you can do, what's happening, how it's all going is important because otherwise you, you know, many of us find ourselves signing up for more stuff than we can do. And without, without having a data-driven conversation, it's hard to—

Correct. Yeah. And I also pair it back to what Chris is saying basically about end-to-end agility and being able to react to new opportunities, right? If we can quickly gauge what the strength of our investments is and has been across different areas, it puts us in a position where we can calibrate and direct the engineering team to new opportunities and we can see where to scale forward and scale back. Equally, last year we invested very heavily in a modernization program around our data center and cloud migration and optimizing and modernizing a lot of our tech, like containerizing and adopting some of the core principles of 12-factor apps, right? Like, we need to, you know, as an engineering team, calibrate how aggressively we modernize and how aggressively we pursue product roadmap features. And it gives us the tools to have that data-driven conversation and to adjust to the business priorities too.

Awesome. I'm keeping my eye on the clock. I do wish we had another hour here. Maybe we could do like an extended cut and put it on the internet or something. But The one question I'd love to ask any or all of you to maybe close us out here, something that comes up a lot when I'm having these kind of conversations is how much the world has changed over the last, you know, 1 or 2 or 4 years. So comparing now to even say just a year or 2 ago, how has what you're trying to do, your priorities, how you're managing and measuring your team, how has that changed? What's different now than, than than a year or two ago?

I'll go, I'll go if that's okay. I, it's a great question. I think that, that, you know, for us, the biggest thing that's changed is, is the business and technology and the engineering teams are more integrated than they ever have been before. And we firmly believe that, you know, there's some escape velocity when you get to that that synergy between those two teams where it's not the business making business decisions and the engineering function acting as a fulfillment function, but where everybody is grafted together and—

Is this happening intentionally, Chris? Like, is this a thing that you're trying to drive or is this more of a trend that is just happening and you're reacting?

It is absolutely intentional. We wanted to sort of blur the distinction between between more applied technical disciplines and, you know, and business disciplines and not ask everybody to strictly stay in their own lane. And I think the decision to move to the, you know, experienced-aligned agile model that I talked about earlier is a big part of that. And driving connectivity into the— across the enterprise in terms of what this means from a financial standpoint, like we talked with with regard to the DevFinOps approach. All of that has been sort of a game changer for us. It's very much part of a broader digital transformation effort. And tools like Jellyfish act as a critical enabler of everything that we're doing and sort of a binding mechanism as well. So it's been It was a thoughtful, planful, deliberate approach that we've gone down, but it's yielded better results than we even originally imagined. So we're thrilled.

Nice. Thank you. Shane or Michael, any big compare and contrast between 2 years ago and now, noting that Shane, we covered, you know, like you've got a much larger team, but anything else?

Yeah, I might add a few things. So there's a couple of key factors that I think has really, you know, if we talk about the world as a whole, you know, it actually isn't that long ago since, you know, the world had to shut down through COVID and really change the landscape of how resourcing and how team collaboration had to happen. Like that was literally 2.5, 3 years ago. And we all had to figure out how we were all going to operate as a business, you know, Now, even with that lens, you know, what does the return to work mean and how do we leverage this new mode and this new way of people wanting to work for us, wanting to build and deliver applications? And actually, our key phrase that you hear in the corridors of Fanjul over the last number of months really resonates with many, many of us. What got us here today over the last 3 to 5 years is not actually what is going to get us into the future over the next 5 years. And actually, that change of mindset and, you know, having more of an innovation transformative perspective into the future, that has to be underpinned by rich access to data. Again, that's where I really feel that the Jellyfish platform, in particular, in particularly around resource management and engineering insights, you know, That's one aspect of how we have to think differently and actually how do we better set ourselves up, how do we better scale, how do we drive more efficiencies. And these are not conversations that we really had 2 years ago because we were too focused on building the business and getting to, you know, that number 1 position was a really difficult thing to get to. It's going to be even harder to maintain because in our industry it's quite It's quite an interesting landscape at the moment with lots of entrants into the market.

That's great. Thank you for that. I think a large part of what you just said is relevant, just as relevant to a 50-person team as it is to a 1,500-person team.

Yes.

Mike, you have the opportunity to have the last word here before we sign off. What would you like to—

I mean, I hard agree with a lot of what Shane is saying. I think the change in mindset is something that resonates very, um, like very well with our engineering team too, right? If I, I think back to how we've solved problems in the past, it's certainly been to, you know, hit the scale button on resourcing and try and figure out a lot of other problems later on. And, um, we're working a lot with data to be a more efficient business, right? Like we have different expectations and market expectations now that we're a public company and we can't just hit the resourcing button as our solution to everything. We have to be deliberate and considerate about how we're spending our time and energy and resources and meet—

There are interest rates now. That's a change from—

Right. Yeah. But it's also important for us to measure success and not try to eat the entire elephant, right? There are so many opportunities for a partner like us for technology partner and ecosystem player like us, we have to be more deliberate in the battles that we do pick and, you know, which battles we decide to partner with other technology companies for or pass up. And data helps us to do that. Awesome.

Well, thank you all so much for joining us here. I think the session was great. I'm hoping that everyone in the audience— thank you for everyone in the audience as well. Hopefully you're getting a lot out of this. And congrats to all three of your companies on winning the Illuminate Awards. And I can't wait to talk to you all again soon. Take care.

Thank you all.

Great.