Guiding the Next Wave Embracing AI's Role in Evolving Leadership and Technical Mastery | GLOW24

Hello and welcome to Lead Track here at Glow for our breakout session on Guiding the AI Wave. I'm Nick Al Brown from Jellyfish Research, and I'm excited for the opportunity to cover this topic that's been in the forefront of our industry over the last 18 months. In that short time, AI coding assistants have begun to fundamentally reshape not only how developers write code and manage their workflows, but how they innovate through complex problems and new opportunity spaces. Today, we delve into how those technologies are influencing the software engineering landscape and what this means for the future of organizations like yours. We're gonna hear strategies from the field about how some companies are adapting to this new technological paradigm, and we'll also cover the bird's eye view of what's happening across the industry. As engineering leaders, how should you be deploying these tools? How do you measure and learn as your organization evolves? What pitfalls should you be looking out for? And ultimately, what does the future of software engineering look like? In order to help us do that, let me first introduce our two panelists joining us this afternoon. Tyler Even is a senior director of software development at Precisely Incorporated, where he oversees a diverse range of projects, including front-end engineering, globalization, and user assistance. Welcome, Tyler. Can you share a few words about yourself and where you're joining us from today?

Yeah, Tyler Even. I'm joining from the Rochester, Minnesota area today. I've been in the engineering field for now close to 20 years. And I'm super excited to be chatting about our use of AI tooling today.

Awesome. Thank you. And next, we have Ryan Murray, the Head of Technology for Thoughtworks Banking, Financial Services, and Insurance Vertical. Most recently, he's led Thoughtworks Engineering Effectiveness and Developer Productivity offering by leveraging experience across platform engineering, developer experience, organizational change management, and more. Ryan, welcome. Can you share a bit about where you're coming from today?

Yeah, thanks for having me here, Nick. I, I have been in the industry for closer to 30 years almost now, and have been an enterprise architect, and have worked a lot in how Thoughtworks helps our clients with cloud-native engineering over the years. And most recently, like you mentioned, we've been focused a lot on developer productivity. So really excited to share what we're seeing across the globe. We have an amazing footprint and see a lot of different customers of different sizes and maturities. And so looking forward to sharing a little bit about what we've seen work, what what we've seen perhaps not work and kind of where the future might be going.

Awesome. Thank you. Yeah. So let's, let's jump into it. Tyler, to start off, can you share a bit about how Precisely is investing in AI coding assistance and sort of what's been the overall sentiment towards these tools?

Yeah, we've been investing in the AI coding assistance now for almost a year. We started off doing a proof of concept using the, the tools that were in the industry, GitHub Copilot, some of the other AWS tools that were available, and some of the open-source-based tooling that we could utilize. Over the last 6 months now, we've been utilizing GitHub Copilot as kind of our standard approach, especially because of the enterprise protections it affords us with some of the IP protection, some of the capabilities, and also just the ecosystem it lives in. So far, there's kind of two sentiments towards it. There's the We understand the power of it and we understand, you know, what it's going to bring to our company. But then also we're also cautious about the legality and the copyright things that are kind of still outstanding and open in the industry. So we've kind of taken a different approach to only invest, use it in certain parts of our product that we feel like we can expose or take a little bit more risk on for right now. But So that's basically kind of sums it up.

Yeah. No, that, that makes sense. And Ryan, across sort of your, your client base, is that what you're seeing in terms of the breadth of companies looking to invest in AI and, and how they're doing it?

There's definitely a wide variety of, let's say, maturity or excitement levels about this. Everybody knows it's important. Most companies that we see are starting in some way, shape, or form with it. And what's been interesting, I think, is not, it's not necessarily, I work now in an industry, I'm a bit more focused on the single financial services industry where I was very, very broad across industries before. And even within financial services, we still— I don't see as much fear of the, you know, kind of copyright, uh, that a lot of people are concerned about. And I think depending on the business you're in, whether it's SaaS, whether you build a product that you ship, obviously there are differences. But in general, I think most people recognize the potential value and that there are good protections coming in place and that those continue to improve. So I find that most organizations are still very excited to try to get the value, the, the difficulty isn't so much getting started. Obviously, it's scaling the results.

Yeah. So then, you know, you know, change management, process implementation, those things take time, right? Especially when it comes to new technology, new tools, like across the market. So what— have there been any barriers to adoption or what have those barriers to adoption been in your experience?

I think the biggest barrier to adoption has been having a program in place that helps people to use the tools really effectively. So chat, you know, Copilot in general, code complete, chat, all of these tools, they work reasonably well out of the box. And, but depending on the type of tasks that people are doing, they see differing levels of performance, if you will. And a lot of people who have been innovating are really working hard to utilize things like chat to get the extra value. Just the pure code complete has, creates the value it creates. And I think everybody's somewhat clear about that. But what we're seeing is that when it comes to innovating on prompting, for example, how can you get more insight? And particularly what we've been really focused on is, you know, be— coding is maybe 30% of most development workflows in most large organizations. And so that gain that people are looking for is it might, even if it were a 40% gain, which we, we haven't really seen on the ground for non-trivial tasks to be really even probably more than 20 or so percent, but that's 20% of 30% of your workflow. And so the real value that we're seeing the innovators starting to capture is how do you use these tools more broadly to go, if you will, shift left GenAI in the SDLC? Like how do you start to build out stories and be able to notice where you might have architectural challenges or be able to notice where there might be security issues that you're not planning for? Those are less mature in the tools and therefore they rely more on engineers learning to use prompting and sharing prompts to be able to get real value. And so I think that that's really the important thing that the leaders are finding is being able to share learnings. You know, this AI is probably one of the coolest things about what's happening is none of us really know what's possible until we try. And as people learn, that sort of build-measure-learn loop is happening right here within the tooling ecosystem. And being able to support that has been something that most companies I think are still struggling to do. Like, how do you, how do you take a product mindset to the rollout of these tools, really learn where they're being effective in the organization, and then build practices and It can be training, you know, prompt engineering is a lot of what we are helping clients to think about doing. Those kinds of, you know, those kinds of efforts are really important to get the most value out of tools.

Yeah. Yeah. I mean, that makes, that makes a lot of sense. Tyler, it sounds, I mean, you guys went through experimenting with different tools and now, like you said, you're sort of slow. You didn't just give it to everyone all at once. You're slow rolling out who's getting it and, and, and learning from that. Can you talk a little bit about that process of, of, of rollout and then how you're, how your folks are experimenting and learning and how you're turning those best practices into sort of a use case for more adoption.

Yeah, no, exactly. I mean, we rolled out to our frontend engineering team first. Part of that was just that, that team has a nice community in our company, you know, where, you know, we come from lots of different acquisitions and lots of different things, but our frontend engineering team seems to be kind of more of a cohesive larger team that already was meeting on, you know, standard practices and things like that. So it was a great area for us to introduce a new tool and practice there. So they meet regularly, talk about, you know, what are some experiences they've had, what have they learned that it's helped with. Interesting enough, what we're seeing, that 20% gain on the 30% was exactly what we were kind of seeing, that everyone kind of thinks this is some magical tool, it's going to like double my development time. What we're seeing is where it saves time is in debugging. Code reviews, and then also learning new code. So when you move a developer from maybe one area of the code to another area of the code, using these tools to explain what's happening in the code, we're seeing significant speed to productivity versus just the standard development practices. And those were learnings from, from engineers trying things out in different approaches. And those are really hard to quantify, right? It's easier, easier sometimes to quantify how fast is it to do a task. It's harder to quantify how fast it is that someone learns something. So like being able to understand those things and have surveys and different ways to measure the success of it is also important.

Yeah.

So how are you like, how are you capturing, how are you capturing those learnings, right? How are you in turning those into best practices? Do you guys have a system for that? Or Ryan, I guess what have you seen in terms of like how to convert sort of what you're learning in real time, right? These changes are happening so quickly and, you know, every time there's a new feature or new subset of the tool, How are you capturing those learnings and turning those into sort of into that change management that we touched on?

Tyler, do you want to talk a little bit about from hands-on how you guys do it? And then I'll talk about what I see.

Yeah, I mean, right now what we're mainly using for sharing the experiences is our just our community, you know, so the development community inside of the company, right? So sharing those ideas just like we would share just finding a better third-party library or finding a new way of doing something. We also are measuring things with surveys. We do kind of like a quarterly survey to the developer just asking them of their experience on using the tool. And if any findings come out from there, we share that back, which can lead to some further conversation. We're just now starting to dig into Copilot released their APIs to get their usage data. Also, you can see suggestions, percentage of used suggestions. You can see the rate of that going differently. You can see the chat interaction grow as they GA chat and different things like that. That. So understanding how we're using it to then ask, you know, prompted questions to the development community is also important. But those are the kind of things we're obviously looking for more ways to track and, and learn, you know, what are we really getting out of these tools, because they're not, they're not free, right? So, you know, trying to understand our return on investment is important to us. And so I think, you know, anything that, you know, we have available to us, we're happy to learn. So Hopefully, right?

I think one of the things that, that we, we've been bringing to our clients is we created a little bit of an accelerator that we originally called our Team AI Portal. And now I think it, the name is changing, but basically just creating a lightweight front-end application that allows people to use chat, share prompts, but bringing all of the engineers to a common place where we can store some of that stuff. So like what Tyler just talked about, it's, it gets so much of this is about sharing what's working. And as the organization size scales, that becomes more and more difficult to do, you know, within just a, a, a informal community. And so having a place where you can go and you can really share learnings and share prompts and share other techniques that are working is, has been super valuable for a lot of our clients. So I think that's important. But the other thing that is super important and most organizations struggle with, right, is how do you measure overall engineering effectiveness? And again, we use this word because we know that like effectiveness is really about value for the dollar. And like Tyler said, you know, we, we wanna really know that we're getting value from our investments in productivity. It doesn't really matter if we've accelerated a small part of, you know, a small part of the code coding process a lot if it doesn't actually yield throughput gains. So having measures in place that allow us to understand the whole software delivery lifecycle and not, you know, DORA metrics as an example, we all know the industry is very excited about them, but they're so, they're so large in the, in the area they cover that they're not very good at targeting and helping you to understand what is the part of the process that's maybe causing a higher lead time or whatever it may be. And so using tools like Jellyfish are really essential to be able to start to get that stronger sense of what phases of my lifecycle are taking how much time, and And now that I've introduced the tool, how do I notice that that's changing? Right? And so being able to really see what areas of the overall cycle time are being impacted by the tool, critically important. And I think that's, that's the thing that we're calling out most to our clients is you have to, you know, the famous you can't improve a process that you, you don't measure. Not completely true. We know that you can do that. But still, when it comes to being able to not only make new investments and further investments in the places that are adding the most value, but also we notice that a lot of the friction and a lot of the waste that happens that drags on our, on our productivity, it isn't always in the easy places where coding assistants might be helping you. Sometimes it's in places like working in shared test environments or other dependencies. And so while this is all about AI, I think in the end really it's all about the SDLC being better. And so I think it's critically important that people undertake this initiative in the context of improving their SDLC. And this is one of the many tools to do it because otherwise what we've seen is some places in the SDLC are being accelerated, but then you break into other throughput bottlenecks. So if you're not addressing code review, for example, then you could be producing code faster and then it just gets blocked at the code review phase, in which case your net throughput maybe didn't change that much, right? And so when it comes to also messaging up the chain to our senior leaders and peers, the CFO, about why we're spending all this money, the throughput is really what matters. The business outcomes are what matter. So being able to tie that all together, I, I think really requires investing in strong telemetry, oh, throughout the SDLC and then you may need to also solve some problems that aren't gonna be solved by GenAI, but that are also gonna make a bigger difference.

Yeah. And I think the important part of what you, you mentioned there is just the, is the nuance to it all, right? It's not as simple as, you know, developers are seeing a 50% increase in productivity. Some of these other more selling points that we're seeing in the industry. It's like when you dig deeper, it's those learnings about the lifecycle, about where your effort is going that are really sort of making the difference here or what, what actually matters at the end of the day, right? More so than like you, you can get caught into this this sort of false dichotomy of, of 50% productivity means we're doing 50% more work, right? When that's not necessarily the case.

Well, and what Tyler said I think is really important about the APIs is this is, we gotta be careful now because Copilot's gonna give us really a window into some very micro optimizations. And it would be very easy to get caught up spending a lot of time of like, how can we get the code complete rate higher or the acceptance rate higher? And that's surely valuable at some level, but many organizations I work with, there are such, there are much bigger problems upstream in their development process than necessarily optimizing that code complete. And, you know, value for the dollar, again, it's, you're trying to optimize the SDLC as a whole, not necessarily just the, the how fast you're typing.

Yeah. So what are some of the other sort of downstream impact things? Or, you know, you mentioned the life, like looking into the lifecycle, you know, we talked a little bit about effort allocation. What are some other of those downstream impacts that you see AI having? And Tyler, for you guys, what are you, what are you measuring? What are you looking for to sort of prove out that it's working, that it's actually having an impact on your software development lifecycle precisely?

Yeah, I mean, I think the key, the key things that we're looking at strictly from an AI perspective, what is its impact on our lifecycle? Number one is we're growing, we're growing. We're growing and we're onboarding, right? And so one of those things that we're wondering is, you know, does this make our, like, you know, speed to productivity of onboarding a new member on the team, does that increase because of the utilization of these tools? Some of the nice things about a tool like a GitHub Copilot or any of those coding assistance tools is they actually do start standardizing the way code is written because as you're getting prompted with standard approaches, the code starts looking a little bit more similar from that aspect. Also, the whole ability to be able to, in context, ask what a code— what a code base is doing. One thing we are already seeing is that the fact that we have the tool available, our subject matter experts do feel like they're getting less questions about the code from early onboarding— onboard employees because they're able to kind of ask those base questions about, you know, what are kind of these— the general— what's this method do or what's this component do from that perspective. So we're trying to measure things like that. But it is hard. It's a hard measurement tool. I mean, right now, my key metric that I use is really developer delight. It's really, do they feel more productive? Do they feel like the tool is happy? How would they feel if we were to take the tool away? To me, it's the most powerful metric I have because I know it's really hard to quantify a tool like this right now. And we are looking for better ways to quantify it. But that's the best metric right now that we're, that we're kind of looking at is, is that does the developer experience improve?

I think one of the things that we, you know, I really appreciate what you're saying there, Tyler, because I think it's important to keep dialing in those measures of smaller parts of the cycle. Like when you talked about being able to explain code and understand kind of where the code is, and then does that take load off your SMEs? That is super important. The ability to summarize documentation, to troll, to search, these are some of the things I think are going to give us actually much more lift in the overall development lifecycle. Because, you know, what we notice, you know, as Thoughtworks, because we're helping organizations transform end to end, And we noticed that in a lot of organizations, people are writing code that often isn't the right code or doesn't need to be written because the upstream part of maybe the story development process wasn't, wasn't accurate enough. And so you end up with something that's been fully developed with high quality, but it still isn't really what the customer needed. And the biggest form of waste in your software delivery cycle is to build really great code that doesn't do what customers need. And so being able to get that feedback, I think another thing Tyler said that's really important is utilizing not only hard data, but also feedback from developers and really getting micro surveys, I think, are something that is really emerging. LinkedIn has talked a lot about how they do this about, and they've of course made immense investments in their own platform. But some of the things that Jellyfish does, I think, can be leveraged for companies that, you know, aren't at that same scale is how do you get the developer in the moment that they've done some work to give you qualitative feedback? But when it's really, when you get that feedback quickly and you get it on a small part of the development lifecycle, the feedback is pretty valuable. Right? If a developer says not just, I feel generally better about this, that's of course important. But if they can tell you, I think I just saved an hour, I just saved 2 hours on this particular task I did. Now that's actually really valuable data. And so to try to just measure, you know, story throughput or whatever is not necessarily the only way to do this. I think really in, you know, kind of in the moment developer feedback can be incredibly important and sometimes more valuable than hard data because, you know, we all know that hard data can get gamed.

Well, and story throughput was one tough thing, right? Because at a certain point they're going to start sizing differently too, right?

That's right.

Small window, right? As they learn that the repetitive unit tests are faster and things like that, sizes started changing, right? So that was a really hard— that was a real realization for us. We're like, oh shoot, we thought we were going to measure that, but then teams started sizing differently because of their use of the tool. So that definitely made it a lot harder.

Yeah.

Yeah.

We also saw, I mean, I don't wanna jump into pitfalls here, but we, we've seen some companies where there's been so much desire to get more productivity from these tools that then the developers actually actively started breaking down work into smaller units, which is a great practice anyway. But they started doing that even before the tool had had the impact. So suddenly people from a higher level are looking down and saying, look at that, we do have a 50% throughput increase. When in fact that, that wasn't true at all. There was no throughput increase actually.

Absolutely.

And I think that actually matches the results that our team is starting to see from Copilot users is, is this, the scope of work and how the work is shaped is actually changing. And that's leading, you know, potentially causing some of these other metrics. So I think that goes, just goes back to having sort of a nuanced measurement strategy and having sort of trying to be micro in, in, in where you're looking and how you're identifying opportunities, potential bottlenecks, but then also having a nuanced like bird's eye view you know, in these, like, as you get larger to, to make sure you're not drawing false conclusions and so start implementing changes based off of sort of a conclusion that's not necessarily, you know, the right one. And, and you got to, you jumped ahead a little bit to the pitfalls, but I think that's a good, you know, can you speak to some of those, some potential negative impacts? So, you know, like we said, keeping a singular, being singular-minded about the impact is, is one potential pitfall, but sort of in the SDLC, like, what are some potential pitfalls that engineering leaders should be looking out for?

Well, one thing that we're definitely seeing, we sort of imagined this might happen a year ago when we were starting to look at this, because Thoughtworks has always been a company that really prides itself on its engineering excellence practices, helping our clients get better at how they do engineering. And we worried that there might be some challenges with coding faster, but more boilerplate type coding. And so what Tyler mentioned earlier is really important, is knowing that your tool is helping you to standardize on practices you want to adopt versus just generating more boilerplate code faster. We are definitely seeing organizations that are seeing a proliferation of boilerplate code where, you know, a senior engineer that, that was more thoughtful might be able to say, I could, I could abstract out something here, I could build a common component. And instead it's like, well, nope, Copilot gave me an answer to this, so off I go. And especially when, when there's, uh, younger developers who are getting, are learning, and, but they're, and they're getting good answers and they're producing code, we have seen technical debt of that sort in, in some cases starting to increase. And so I think that that's, it's really important to, in, there are different ways to do this, but important for organizations is to know that the long-term codebase health is still something that they have to care for. And at this stage, I don't think any of the tools are able to really support that directly. And if anything, they can push you in the opposite direction of just generating more code faster. So I think that that's an important thing to pay attention to. The other thing that we've noticed is, like we said a moment ago, building more code faster means more stress on potentially on code review processes, more stress on CI/CD. You know, if we like to say, if you've made investments, like GenAI will amplify whatever it is that you've got. And if you've got a really strong, like downstream SDLC so that your tests are highly automated, you're able to ship code fast and get feedback, then this can absolutely amplify you. But if it doesn't, you start piling up a lot more code that's waiting for, you know, reviewers and other things, and that doesn't yield throughput gains. And it actually can start to cause the code review process to be less thorough because there's more code piling up. So again, noticing the behaviors in the whole system is very important. And this is no longer hypothetical. We are seeing teams struggling with almost like generating too much code too fast for other parts of the SDLC to keep up.

Yeah, Tyler, how have you guys, or how have you managed sort of that, or what, you know, what pitfalls are you potentially seeing sort of as your developers have, especially as they've gotten more comfortable with the tools as well?

I mean, I think the good news there is, at least in our, you know, in our standard practices, you know, we have those protections in place right now. If we're stressing those different places, that's where we find out, you know, you know, we can look at, you know, the average code review time, you know, lead time and cycle time. You know, we can look at those things, but I mean, utilizing things like static code analysis to understand, like, do we have a bunch of duplicated code now? You know, we already have protections that try to eliminate, you know, duplicated code from even entering into the code base, you know, using tools that scan the code. And, you know, hopefully, you know, that starts giving you those best practices of what you're mentioning. We definitely have seen with the increase in the ability to write code faster We are seeing the code review process, you know, feeling more pain, but we're also— but it's hard for us to like, is that just because AI or is it because we're growing and we have a bunch of new members, right, that are maybe junior members or new or new to the team and aren't approvers yet, right? From that standpoint. So it's kind of hard for us to quantify just because we have a couple of balls in the air on that side. But we just really have to follow like what we've been talking about all along today. It is the bigger picture, right? You have to have those checks and balances in place to ensure your code base is solid. And, you know, we'll continue to look at it. I'm really excited. Our next journey with the AI tools is all of these AI ecosystems are moving into then help those other areas of the SDLC. You know, how do you make merge requests better? How do you do backlog refinement better? Things like that. And we're really excited to start dipping our toes into those as well.

Another pitfall I think that we've seen in some larger— Tyler's obviously operating a pretty tight ship there from what I can, from what I'm hearing, and that's great. But a lot, a lot companies, you know, don't have that engineering discipline already in place. And so I think that there's, there's a lot of, there's a lot of risk, if you will, also to towards if you have a, if you have a complex legacy codebase, right, and a lot of the work still requires real thoughtfulness about how to, how to deliver the work, then again, the benefits are gonna appear smaller, right? If you're in the, in a state where your codebase is either because of the nature of the code you write, you're able to build new features that are kind of additive and you're not constantly reintegrating or have challenging, you know, integrations to downstream systems. And there you're gonna see the benefit much more clearly and be able to measure it and show it. I think I would be careful though for organizations to really pay attention to not setting expectations that are too high, really managing these expectations. Because, you know, when, when I was first talking about this with colleagues and we were hypothesizing that the gains weren't gonna be 40%, they'd be a lot lower. One of the things that a few of us said was even if the gains are 5%, they're probably still totally worth it, right? So what you don't wanna do is have that kind of shock where I thought we were getting, gonna get double our speed, we're only getting 5% and we're not even sure if it's 5%. So forget it. Like, let's start, let's go and invest in other tools. Like, you wanna make sure that you understand the nature of the code that's being written and you, you set reasonable expectations for what the AI tools can do to enhance it. And I think we'll, again, as we go upstream more and we're able to use the AI tools to look at architecture and propose things that maybe would've been missed, that can really smooth out the early parts of the SDLC. And so for complex environments, those gains might be more valuable than, you know, than the code generation side. But again, really engineering leaders, I think, have to be thoughtful about what kind of gains they actually think the difference tools can get for them and make sure they set expectations correctly so that you don't end up with a big letdown. I, I will say that in one of our major financial services clients that's considered one of the most tech-forward kind of banking companies out there, you internally, you know, I, I heard recently some of their AI gurus saying, ah, this is at peak hype, we're, we're not sure we're really going to get the benefit from it. And so we're actually starting to hear some backlash, you know, about, hey, this is all too hyped, we're not getting the results. And I think that, that the risk is throwing the sort of baby out with the bathwater, if you will. Like, we, yes, there's been a lot of type, of course, but you don't wanna throw away a 5% gain. You know, that's, you know, the licensing costs are not that high. Like 5% still would be a fantastic gain to get.

Yeah. No, that's a great, that's a great point. And I think we got a couple minutes left here, and I think that's a perfect segue into talking about how the job is changing, right? You, we, we, we're talking about all of these changes that are happening that are rapidly, like, to, to the SDLC. Like, how are the requirements of the job of a software engineer changing? and what technical skills, like what does the future look like, uh, for software engineering? So maybe give you like about a minute each to, to, to give me sort of your spiel on the future.

I can start.

I, I mean, I really don't see it transforming what a software engineer does. I see they just will work differently, right? It's no different than when they used to have to punch, you know, code cards in and, and walk them up. I mean, the, the whole aspect of being a software engineer is to use software to solve problems right now. What these tools just allow us to do is hopefully do that more efficiently, you know, maybe take some of the mundane, repeatable tasks and just do it for us. But we're still focusing on how do we, how do we solve a problem with software. And so I really don't see it fundamentally changing the software engineer, it just will be a different tool that we'll use in our toolbox.

Yeah, 100% agree. But I think the thing to call out is, you know, great software engineers have always been great problem solvers, they've always been thinking architecturally and thinking about how to drive efficiency over time. Like, how do how do you sustain momentum and have a code base stay healthy? That aspect has to be, I think, taught more because a lot of times we are focused on what's the latest framework, how do you use it, the mechanics, especially as you're coming into the industry, there's gonna just be a lot of how do I get stuff done with React versus, you know, whatever it might be. And, and I think that we're gonna have to probably really put more attention on what does it mean to have, you know, solid principles, great code, well-factored code that may need more attention. So it's not a fundamentally different job. It should have always been there. But I think more and more, if it's not there, you're gonna see a more rapid proliferation of tech debt, which simply means losing your momentum and losing your speed to market, right? And so that's probably gonna need some more focus in all forums as we, you know, as we bring people into the industry and we help upskill them, that, that, that real architectural thinking and that long-term thinking is gonna become critical.

Yeah. Well, we're coming, screeching up to the curb here. I wanna thank both of you so much for your time today. I certainly learned a lot, and hopefully the audience gained some tactical insights from the session so they can apply as they go back to their organizations. And, you know, we see these tools start to become more and more ubiquitous. So thank you everyone and enjoy the rest of your time here at Glow. Thank you, Bob.

Thanks for having us, Nick. And if anybody, you know, finds they have any questions, don't hesitate to reach out. We'd love to help.

Yes, thank you everybody.