Scaling Tech Operations Without Compromising Growth with Andrew Parsons
13 Nov 2024 · 74 min
Andrew Parsons, who scaled operations at HSBC, Mastercard, and Admiral Group, explains how to grow fast without breaking things. He warns against over-engineering and too many product tweaks, and argues that clear goals, roles, and accountabilities matter more than speed alone. A UPS test failure and a data-center flood taught him that automation and risk management are what separate companies that survive scaling from those that don't.
Chapters
Picture this: your tech venture is scaling fast, but behind the scenes, your operations are chaotic. Resources are, resources are stretched thin, and processes are breaking down. How do you keep the momentum without everything falling apart? What strategies can you put in place to ensure operational excellence is driving, not stalling, your growth? Welcome back to Mastering Tech Growth, where we interview industry leaders to bring you actionable insights and inspiring stories. I'm your host, Mike, and today we are diving into operational excellence, one of the most critical but often overlooked aspects of tech growth. Whether you are scaling a startup or managing an established venture, Your operations need to keep up or your growth will slow down. Joining us today is Andrew Parsons, a leader with decades of experience transforming tech businesses. From launching Vivebank to scaling tech operations at HSBC, Mastercard, and Admiral Group, Andy has mastered the art of turning chaotic operations into growth enablers. If you find value, in the show so far, please don't forget to subscribe, leave us a review or comment of Apple Podcasts, Spotify, or wherever you're getting your pods from. Let's do it. Welcome, Andy.
Thank you, Mike. Yeah, thanks for having me. Looking forward to the, uh, to the conversation today.
Me too, Andy. Operational excellence is, you know, just as mentioned, so highly overlooked when it comes to enabling the growth. So before we dig into the weeds of all of it, We have this question which we love to ask in the show. When I say mastering tech growth, what comes to mind?
So my background is predominantly financial services. So, you know, tech growth for me is about, you know, exploiting what's new and trying to get that into the marketplace for the benefit of customers. So I think the mastering part of it is really getting to understanding, you know, it's very easy to choose lots of new products and chase after the shiny new stuff. But I think fundamentally for me, the mastering part is, get underneath the skin of it, make sure it aligns with your strategy, make sure that you can see the benefits of it, and then go after the things that make sense and scale it at an appropriate rate rather than just big bang. So I think, you know, coming back to my point about, you know, my career has been in financial services and we're always very good at getting after the new stuff very quickly, but I think there's always a kind of air of caution about that and making sure that you don't you don't take big risks. And fundamentally, the mastering part is about risk management for me.
Okay, so if you— if, if I would rephrase the question and say you need to choose one thing from all the things available when it comes to tech growth, you would choose the, the risk management of any sort of like initiatives?
Yeah, I think there's a matching principle here for me around understanding. So, so I had a boss many years ago, and he— and his mantra was, you know, understand the ballpark you play in. And so to me, I work in financial services. The whole premise of banking is that it's somewhere that you put your money and it's safe, secure, and part of a system that allows for some growth. But I think risk management underpins almost everything we do. You wake up every day thinking about risk management. So part of that growth is being able to grow safely in a compliant way, making sure that, you know, the premise of the banking system is safe and secure. And then that will then therefore keep its reputation. So I think, yeah, risk management is an interesting subject. I think, you know, if you work in financial services, and I have done for, you know, two and a half decades now, and I spent some time in the military where you are, you know, wake up every day thinking about risk management. You think about what could go wrong, not always in a negative way because you can exploit that aspect. But yeah, risk management is, it's the daily day buzzword. You're always trying to do something to make sure Is it the right— is it the right thing for now? What could go wrong? How do I protect my— you know, all paths really lead to reputational and reputational damage and financial censure, if you like. So yeah, that's, that's the way I always think about the growth aspects.
Okay. Can you over-risk manage and does impact your growth? So you're becoming you're not onboarding enough risk to be fairly innovative and, you know, and choosing the tools or going after initiatives which potentially have, you know, huge growth and you're becoming too risk-averse and as a result potentially stalling your growth?
Yeah, I think that, I mean, that's always possible that you, you know, there's the sort of, you know, you can take a cautious approach and do the research and do the assessment. So, you know, identify, assess and understand what it is that you're getting into and make those choices. I think some good examples, like, you know, take core banking systems, for example, a critical part of your estate. Everyone's got a core system of some description, and there are lots and lots of options on the market these days. And you can, you could choose, for example, you could choose something that's really popular that everyone is using, and there's some obvious ones out there. But the risk is, so the benefits are lots of people are using it, there's lots of investment, it's proven to work. But the risk management part is about thinking through, everybody is using it. So how far down the list are you, or how far down the queue are you in terms of getting changes to that product set that you want? So again, kind of thinking risk here. You can choose the products that's the best on the market, but it might not be able to scale and grow at the rate that your growth ambitions are set. So the alternative is you go for— and I've had this experience here, so you go for a product that not everybody is using, that is subtly different, but you end up in a situation where that vendor is kind of almost explicit to you and you can really start driving the changes and the innovation at a business level that you want from that product set without lots of a queue of other large companies also trying to place their demands on that against the market, you know, set of market conditions that we're all, we're all trying to compete in a in quite a small space, actually. So there's a kind of a bit of an example of, you know, actionable insights is have a think through, you know, when you're choosing your new products or you're thinking about getting into a new area of innovation, you know, what does that really mean when you get into the detail of scaling and growth? Hmm. I love it. Not everything. Yeah, not everything. You know, you don't have to go fast, but you do have to do it in a safe, controlled way. I always think for financial service. Now, there are sectors where You've got to be at the market very quickly. You need to be at the front, and you have to, you have to weigh up those risks. So fundamentally, back to your question, yeah, it is about risk management and weighing up the pros and cons of what you're getting into.
I love that, Andy. It's a very good shout. I just, I think Jeff Bezos has a rule which he calls like a two-door choice rule or something like that, where, you know, in any given situation you have two doors, you know, you can walk through one, but you can sort of walk it back And another is you walk through that and it just shuts and you cannot take it back.
Yeah.
So when you're making—
It's the sort of lesser of the two evils, isn't it? So if you've got two doors, yeah, well, so your first, I think the first rule is what happens if you do nothing? And you know, that's such an easy start point, but in, in, in, in the world of tech, and actually tech is still fairly new if we, if we talk about it in the last sort of couple of decades, it's really accelerated, but And you could go back a bit further where it was probably a bit more dumb terminal type technology. But I think in the modern era, you know, everyone wants to scale and it's very easy to scale. There's lots of great products out there. You know, cloud just allows you to do things really quickly. And again, back to the risk management scenario here, the cloud. So I, you know, so we built Vivebank in the cloud. At the time it was so 2018, 2019, it was the first 100% cloud-based retail bank. So lending and deposits. And, you know, some of the conversations with the regulators at the time were around, so around, you know, is it safe, will it scale, is it resilient? It's all of those questions that everyone was worried about with cloud right now. You roll the clock forward to today, 2024, it's really difficult you'd find anyone in a room saying, why aren't you putting it on the cloud? Why would you go on-prem? So, you know, the other aspects of scaling and growth is if you choose the right you know, if you choose the right hyperscaler, you can get there very quickly. And I think there is a danger to, you know, you can get there very quickly. And I've seen and I've heard, you know, myself, difficult to do a podcast these days without, without mentioning the word GenAI. But, you know, there are some great language models that are being built up and there's some great work that's been done, but it's consuming huge amounts of credits. And people are doing things very, very quickly and suddenly finding that they've blown a year's worth of I'm going to put, you know, funds on something I haven't really got anywhere. So, you know, I think risk management for me is the fundamentals. As long as you are clear on what risks you're willing to take and also have your mind on how do I roll this back. So your door analogy there is quite interesting, is that ultimately you've got to walk through one of them. So which one of the two is least risky? And then there are some parts of our economy where people will want to go through the most risky door because the time to market is crucial. And I think when you work in financial services, there's a real FOMO, there's a real fear of missing out sometimes when you sit there in the technology and you sat at the top table and you've got the board pressure asking you, you know, are we going to get into this? And, you know, I've had to learn to take a deep breath and say, okay, what's the right thing? And bring all of your team around you. And there's always sensible people that will, you know, you'll have a good mixture, hopefully, if you've got the right team around you. So some will say, yeah, let's get after it and get all excited. And typically they're going to be much more product-related, I think, from experience. And then there will be, you know, security people and others, risk people, who are saying, hang on a minute, let's think clearly. And I think personally, I've always sat on the side of, I love the idea of moving quickly, but actually I want to do it in a very controlled, structured, standardized way, which actually sounds, you know, it sounds boring, but actually I, I think you get a better outcome from that ultimately.
Well, let's talk about this, Andy, because moving quickly for me, it's effectively operational efficiencies or operational excellence. It's, it's being able to go from A, you know, to B, where A is we have this idea, B that idea is in the customer's hands, how can you close that distance as fast as possible, but in such a way that you don't onboard too much risk, so you don't cause problems eventually for yourself, everything done in such a way that it's a maintainable piece of feature or product or a service. So from your perspective, if we look maybe at a little bit smaller scale, not at big enterprises, but if we think of startups or maybe a startup which is close to becoming a scale-up, So things are chaotic. Maybe we just raised the money, you know, maybe we just went through the Series A or we got some type of seed. So that, you know, that growth will, will just be very inflicted very, very fast, right? So like one day we have 60 people, next day, you know, we got the funds, we now need to be 120 people. And, you know, that changes everything. In your experience, like, what's, um, like, what is the most gotchas you should be looking after?
The gotchas, the kind of, uh-oh, I didn't realize that. And they sort of, oh no, I think we've gone too quickly. Yeah. So if I look at this, so if you think about big enterprise, actually lots of, you know, I've worked in a few tier 1 banks. They are just massive companies that are thousands of small companies, actually. So actually, even if you think about a product, if you're bringing a new product to market, you know, in the olden days, I wasn't that long ago, actually. But, you know, we would just do— we call it test and learn. I mean, these days we call them MVPs, or we call them a proof of concept or proof of value. I think get into that mindset of like, how do I build the minimum viable product I can right now using the best and the cheapest tools that I've got? Prove that you've got something worth pursuing. I love that. I love the phrase, you know, fail fast. I think that's really important to have that mindset of if it's not working, be brave enough. So I think You know, that sort of strategy lends itself to certain types of characters who want to do that. So, but I think, you know, in the backdrop of that, you still do need to have some control. But the gotcha moments are often, you know, my experience here now talking is you move along that journey of MVP and you have too many, you know, too many product enhancement, product changes in trying to do too much too soon. And you end up with a bit of a diverse kind of view of what you're trying to do. So I'll give you an example, you know, building a mobile mobile app that you're trying to, you're trying to build at the same time as building your retail bank from scratch. And actually, you know, the gotcha for me in that example was I've got a mobile app that doesn't actually convert, it doesn't do any conversion, it's just a nice to have. And I think it's, for me, the gotchas are really about, you know, it's leadership and communications, it's making that it's making that conscious step back to say, uh, are we staying true to what we originally said? So if you're trying to build a retail bank as an MVP, is the mobile app really going to help you? Is having every type of lending product and every type of attribute to that lending product really important? It just— do we just need to get out there and deliver against the specific business requirement or the reason that, you know, everyone invested into it in the first place? I think on the team side of it, for me, I have spent a lot of time in big corporate environments where, you know, you have to think about the sort of big scale and the big impacts. And when you're in a small business, you do have to think about that as well, but you just don't always have the funds. So I think hiring the right people at the right time is just— it's a real skill that you have to learn to develop to say, how long can I keep going without 5 security people? And how come— how can I last with 2 or 3? And I think there's a natural point you get to where the chaos that you've managed to wrangle starts to become unrecognizable from the sort of original, the genesis of your original ideas. So I think there's always a natural point when you start looking at, you know, trying to optimize and standardize your processes. I think that's the point where you've got to, you know, you've probably moved from being small outfit to, to something that's more recognizable as you go through that series of scaling, you know, you know, A to, to whatever the funding cycle is. So I think, I mean, the gotchas for me are, you know, they're always scars, aren't they? They're always kind of, I wish I'd done that differently before, or I wish I'd taken a different path, or I've just been able to react to the pressure from investors, from board members, from people who are invested in, in making sure that you carry on with the, the original premise. But, uh, yeah, I mean, the gotchas for me along the way, you know, too many scars. Um, but I, I always come back to, you know, I've always tried to be, you know, clear goals, clear roles, clear accountabilities. And if those are all correct, the people with those accountabilities will be the first ones to come back to you to say, I need another person. And, and it should be pretty obvious because they'll align to the roles and align to the goals. And that's been how I've sort of tried to look at it. It doesn't work perfectly every time, but, you know, having a bit of a framework or a bit of a, a, a, uh, some principles to work to often helps.
I really love how it sounded. Clear goals. Now, clear roles, clear goals, clear accountability. It's almost like a, you know, like a phrase you can put it on a wall.
Yeah, it was easy to remember, right?
It's one of the 3s.
But I think if you get those right, Yeah, like, you know, certainly as a leader, you could, because you can't do everything yourself. I mean, we like to think that we can as a CTO or CIO. I'd like to think I have my hands on everything. But, you know, learning that, you know, when you, if you're a CTO or CIO, you just, you're just not in enough detail, but actually, you know, kind of really between the two. So setting out clear goals, clear roles, clear accountabilities, communicating your, so I've done this a lot, but, you know, communicating every day what are the priorities and then being clear when some of those have to change and why. Yeah, people love attaching themselves to this, you know, this is where we're going. And actually, when you, when you at the start, and I remember, yeah, there's a few examples in even in big, you know, tier 1 banks where you start projects at the start and so a couple of you in the room staring at each other saying, I don't actually know how we're going to get this going, but let's just take the step anyway. And you build from there and you get the right support around you. I think being willing to ask questions along the way. So I think the gotchas are in the early days, they are fail fast, figure out that you've overspent, put your hand up and admit it. I think if you're in a small business that's a sort of fintechy trying to move forward, I think you have to be really careful where you spend your money and who you partner with and how you go to market and make sure it's it is what you say it is and you're not saying it something that it isn't, if that makes sense.
It does. Um, the communication piece, Andy, I, I, you mentioned, you know, it's important to communicate and maybe even over-communicate because things tend to change quite often in the hortic environments. And that is my experience as well, that the rapid change when you are In that, you know, growth phase, which are often driven by mergers and acquisitions, you know, you as well mentioned that, you know, big banks is effectively just a bundle of smaller companies. And, you know, that's my experience as well, not necessarily, you know, with the fintech, but when you have big companies, they, you know, they tend to buy smaller companies. So, so that growth is often fueled by mergers and acquisitions. And then, you know, when new companies are getting bought, they need to be sort of brought into, and it just opens up who's doing what, you know, like who's gonna have this role, who, you know, it's just creates so, so, so much questions. And as well, that change of priorities is very often as well. How do you, you know, sort of inject some type of like process optimization or operational excellence literally to that fact of the, you know, the often changing priorities? How can you make this better?
Yeah, I mean, there's a couple of thoughts here. One is if it's around technology, there are so many great frameworks that you can attach yourself to. If you're a developer, there's lots of frameworks there. You can say, right, this is, this is how we build and ship code left to right. Here's all the toolsets. So if you can align those and get people, and you know, even if, even if you, even if you're sort of merging two companies or you're acquiring a company and they come in and say, well, we use a different toolset, be brave enough to say, you know, kind of which one's the best of breed and we'll move on from there. Maybe you'd have done that through due diligence. So at a technical level, there's, there's frameworks. So we were talking about operational excellence. I remember doing ITIL when it first, you know, 2 decades ago, ITIL, the IT Infrastructure Library. It's a good framework. You can use as much or a little as you want, but it gives you that good sort of— so the original one I did was a few years back now, but you know, it was service delivery and service run. And so, you know, attaching yourself to those frameworks that allow you to be consistent with how you deliver and how you manage and how you deliver change. So I think that's one aspect. On the people side, you know, when you acquire another company, and I've only been through this a couple of times, so HSBC was very good at sort of, you know, you would acquire lots of other businesses and they would come in and I think sometimes their strategy was allow people to run for a few years so they felt like they were sort of part of the company, but they weren't fully integrated. And then over a period of time, you would then integrate what seemed like the right thing to do through, you know, analysis. I think if you try and integrate too quickly, you know, those cultures coming together is really sometimes quite challenging because it can look very good on paper. And I've seen this a few times. On paper, the cultural alignment looks perfect. And then on day one, you kind of realize you're divided by a common language. And yeah, I've been through that. I've been through one big acquisition where It was very positive and it was great for the market. It was a great outcome for everyone. But, you know, sometimes the first 6 months were bumpy as we tried to look at, well, you know, you have this process in this company, but in the company that's just acquired you, they don't have it. So where do you fit in, you know, where do you fit in into the big picture? I think what's really hard as techies is we're very good at the technical language. And I think, you know, you mentioned it early, I believe I you can never really communicate enough. You just got to keep going and find the, you know, find the 3, the 3 bullet points that you can keep playing back and give people some comfort and confidence in, you know, in what's happened. But, um, yeah, there's a few, there's a few examples where I think that the technical change, the perpetual change, is something you've got to get used to. You're never going to, you know, if you work in financial services the regulatory change that is driven by customer demand is never going to go away. We will all, you know, you know, you and I have bank accounts, we use the banking system, I would assume, but you know, we use the banking system because it's good for us, because I don't want to carry around a bag of cash. But what comes with that is, is the response to the changing in regulations to allow consumers and customers to get better outcomes for themselves. So, you know, if you work in technology and you work in financial services, you are under a regime of change, and it will never stop and they come at you thick and fast. They are often, uh, it will take you 18 months to go through the, the point that a regulation comes into force— sorry, is agreed— and then 18 months to when it actually comes into force. So you've got that 18-month gap. Now logic says, well, you've got 18 months to assess it and see what it looks like. The reality is you're already dealing with a load of stuff that came through in the last 10 years that you've got to then move through the system. So I think keeping, keeping your teams aware of what's, what's happening, communicating change, communicating the need for it and why and the whats and the wheres and how that fits in is a big part of the job. Actually a lot bigger part of the job than I ever realised going from being an engineer to leading multidiscipline teams. You've got to learn to communicate differently. So the example is, you know, I love my development teams because they come in, they put their headphones on, and they are— they get in the zone and off they go. Now, you know, if you are— use security as the example, you know, talk to security guys who are seeing, you know, alerts and all sorts of threats popping up every day, that they're not— they do not have their headphones on. They're often very different characters who will confront you very quickly and say, I need this, you know, I need you to make a decision now. And so again, all of that is mixed up amongst change. You know, you've got one group of people in development who are trying to write, you know, you know, applications and code and manage that, that change. And then at the other end, you've got real live incidents going on where if you're not careful, you could get dragged into every single one of them when you don't need to be. I think the backdrop is like communication and managing change is the perpetual, you know, that is something you have to learn to deal with and be really effective at being consistent. I think I've found the best leaders I've worked for just being consistent with their communication, but also very calm. And I love— there's an example of a CEO I worked for who, you know, before we went into board meetings, you know, would always say, you know, what are we going to get into a tangle in? You know, we'd have stacks of papers and we'd be all ready for this, for the grilling. And he'd say, okay, let's just talk about what are we going to get into a tangle? Let's talk about the person. He would go around the room very carefully for each person and let them talk and they go, okay, thank you. And then we'd go into the meeting and more often than not, all the things that we had talked through, we would have been able to deal with without the massive stack of papers. I went off on a tangent there, but yeah.
Hmm. It's okay. I think all what that leads to, in my view, which is very often overlooked, especially in the bigger enterprises, is ability for leaders to actually manage change. And change management as a skill, you know, I see that in many CVs, but when you actually talk with people, like, what's change management in, you know, in more like actionable items, people don't really have that science behind it. You know, everyone agrees that communication is important, you know, setting real goals, maybe OKRs, but there is more science behind change management to actually make it a good one. And I mean, we could talk probably like a separate episode just about like change management. Um, coming back to what you were giving Andy there, you mentioned consistency around the frameworks of how you deliver value, which of course, you know, Agile is, is a very big one, you know, Scrum, Kanban, and, and some spin-offs of, of that. But ultimately, combination of the above. Yeah, exactly. So ultimately, people do have this, but What I get, and I see that very often, especially in a, you know, in smaller startups where the business very often pull rank on what the developers are doing, because you walk into that room where that, you know, change is so rapid, right? So you are, you know, eye of a storm of a change. Maybe you're adding clients, you know, day in, day out, maybe flagship clients, you know, that they ask for things. And suddenly, let's say, you know, you are following something like Scrum and you have biweekly sprints. As a developer, you know, you love that consistency, you love that clarity. I know what I'm doing for 2 weeks, you know, this, I'm, you know, delivering value. But the business comes in, they go like, no, no, drop this, start that, you know, you're starting this, drop that, start that. And then, you know, suddenly there's a lot more you can choose. So you have this huge mental complexity shifts which you need to do, you know, all the time, and it becomes very unpleasant. For you as a developer, you know, to sort of, uh, do those things, and for you as a team lead to actually lead such team. Do you have any tips of like how, you know, team leads or leaders of teams could actually talk with business to, to make sure that there's a, you know, there's effectively some excellence within how business gets things done for them and they don't, you know, cause all of this chaos? Because we all—
yeah, I think, uh, yeah, it's such a great example of, you know, Agile with a capital A or a lowercase case A, you know, is it waterfall? Is it something else? I mean, fundamentally, you're writing code to do one or two things, right? Build a new product or fix something that was already there. So, you know, and even if it was build a new product, it's built it from, from the start. So I think, you know, in the world of development, I think it's really, it's really hard to, to sometimes be able to do that context shift. But you look, if you work in financial services, that is just something you're going to have to deal with. The world of regulation will drive things at the pace that the regulators want it. And that's a good thing. I actually don't, I would never want to see it as a bad thing. I mean, maybe it's too much regulation, but I think in terms of, you know, wisdom for team leads, it's really important to get it, to get your teams aligned to what it is, what's the big picture, right? And keep that focus on what are we collectively trying to do? Technology for me is, it's a team sport. It's about what we're all trying to do individually. You're like, I don't know anyone that can do it all individually on their own. And I think as long as teams buy into, if I can get, you know, 80% consistency with the messaging and the sort of workload and whether you're Scrum or Kanban and everyone's got their preferences. I'm, you know, personally, I'd like less Scrum. I like more doing and less talking. But sometimes the, you know, you have to get into those different sort of mindsets and those different environments to be able to get the right output, the right outcomes. But I think fundamentally, there's some— there is some investment needed by, you know, if you want to step up from being an engineer to a team leader, you've got to be really good at communicating. And I think communication is not just you being able to talk to your team, you need to be a really good listener and an active listener. And there's lots of work that you have and time you need to invest in Being in the room and being able to kind of, you know, read between the lines, understand the body language, understand the sort of message, and think about the questions that you're asked. I mean, at the end of the day, if you're going to lead people, you need to be able to deliver a framework that they can work against and go off and do that on their own without any input, or you're going to have to at some point get in the way of their, of their, you know, work zone and say change of direction. This is why, this is what I need from you. Here's how much time we've got. And if you keep that, you know, clear goals, clear roles, clear accountabilities, right? Keep it really easy for people to understand. But I think, you know what, it really is, it's really difficult to context switch. And I think a big part of my advice would be don't fight it. You have to accept it. It's a bit like change, you know, if you don't like change and you want to go and work in financial services, which is a very wide statement, right? Financial services is massive. Let's say medicine or engineering. But, you know, it's perpetual, it's always going to be there. So I think if you can get used to and get used to and accept— now, I think where it, where it can get tricky, and I've, you know, plenty examples where, you know, I've spent a year working on something, and then just when everyone's comfortable and we've done all the work and we've got the funding and we've got the approvals and everyone's happy, you know, a decision from from somewhere for the right reasons normally is, yeah, we're not doing that, we're doing something different. And I think those are— that's very different because that can cause frustrations in that sort of— you can get a bit sort of negative around your thought process. But yeah, I think it's really important to manage expectations. Ultimately, there's tough conversations, and what I've tried to do, you know, as a leader is have those tough conversations. Now, there's one, one point is keeping your team happy, and you can use a lot of energy. And I definitely, I'm definitely guilty of this. It's like, you know, you know, vigorously fighting for my teams, trying to get the right thing for them. And then you sort of butt up against the, the sort of the rest of the leadership. But yeah, I don't think that's a bad thing. I just think, yeah, everyone's got different ways. You could be— I'm not saying it's right or wrong, but you could be the opposite and, you know, be, be someone that says yes to everything, and that will cause frustration. So You know, I think if you're going to find, if you're going on that leadership journey, and you want to step up from being an engineer, you're gonna have to accept that you're probably not going to be touching the keyboard as much as you used to. And that time probably is worth investing in understanding your communication style, understanding how to, how to be a good listener, and read between the lines and maybe pick out the questions that you want to ask and find a way of asking them that's not going to, you know, negatively charge the conversation in the wrong direction. Trying to find a nice way of saying, don't make it, don't make the situation worse. But sometimes, you know, a bit of conflict is not a bad thing either.
Yeah, you can live without conflict. It allows ghosts in the closets to come out and then you have, you know, them on the table or you dice everything.
What I love about technology, and so anyone that's worked in technology, technology, you know, broad, broad statement, we're all moving forward. We've all had to learn new ways of doing things. And I, I can go back to start my career where, you know, you know, on a trading floor in HSBC, really busy, long hours, long days. Most of what you need to do is out of hours because you can't do it during the day. So during the day, support, I mean, it was really kind of, you know, it was very much a you know, agile environment, then you would, you would have to do everything. You'd have to be, you'd be applications coding, networks, database, you do everything. You'd be crawling under desks doing cabling. And then the world changed. And actually, you know, even now today, as I look back and think, you know, technology is still relatively new, but actually a lot of what I had to do and manage 25 years ago helps me better at being in the room when there's crisis. To be able to go through that logical layer of saying, so what happened at the, you know, what happened at the physical layer? What's happened, you know, if you start at the sort of application layer and the presentation layer and work all the way down through that, that experience starts to shine through and you can start probing. So, you know, we've all been on the journey of change and I love the fact that that's what's so great about technology, working in technologies It's always moving and there's more to go. There's, I mean, there's a whole new subject we could probably get into around, you know, what's the future going to be look like. But I think that's the exciting part of it. And so to any, to anyone that's, that's wanting to make that shift from being a, you know, you know, first jobber through to leadership, you've got to go and invest some time in, in your leadership skills. And I think they really come down to being a good people person and being good at communicating two-way.
I love that. Not everyone is a good leader, you know, but everyone can become one if they wish to. Um, can I give you this ending? And it's very interesting to me how you would solve this. So I've seen this picture where money comes in, um, and business wants to scale. They want to go from startup to scale up, you know, and they want to add a big amount of headcount. But there's a real talent shortage. So that means they cannot really fill the seats as quickly as the plan is, but they still need to deliver, you know, to the promises, uh, for which they got the money for. So effectively, you know, it's a— it's this race condition where the investors expect the features comes out, you know, more market shares being captured, but you really don't, you know, unable to fill the headcount to drive that growth. How, like, what's your approach? How, what, what would you do in that situation?
So a few examples I've been through. I think when you are in a situation where you do need to scale headcount, you need more people to deliver more products and features, you've got a couple of options. You go into the market, you try and hire your own people, and there's, and there's, there's an overhead to that. It takes a bit longer. Uh, you probably will get good people in if you've got good processes and you'll, you know, you'll retain some of that IP for the amount of time that that person stays around and you'll give them a career and they can get invested in the company. So, right, that's the easy choice. I think the other easy option is the world is full of brilliant integrators and partners and ecosystems that allow you to scale very quickly. So you've got the, you know, onshore, nearshore, offshore, there are different ways of describing, but there are many ways to scale your teams. Is short-term or long-term. And I think it really, it's, for me, it's less about employee, um, FTC or fixed-term contract, or, you know, onshore, initial offshore, whatever that model is. I think that's, that's the easy decision. The hard decision, I think, is about for how long. How long do you want to be baked into? So I think if you, if you look at your product roadmap, that's one way of looking at saying like, you know, we've got 2 years, we want to build this much of the feature set, we want to, we want to get out of this series of, of scaling, then there's a bit of a matching principle there again where you can look at different ways to augment your teams. I think the other aspects are, um, yeah, don't let the money burn a hole in your pocket. You don't, you don't have to hire everyone. I think you can, you know, a good CFO will allow you to look at how you can do that over a longer period of time depending on where you are, depending on where your well, we'll talk about financial services really. So where, where you are licensed from a financial services product perspective, you might be able to use different accounting rules to be able to offset CapEx, OpEx. So I think, I think the trick to it sometimes is talking to your financial, to your financial, if you've got a finance team, to say, you know, what, what's my best options? What, what's going to give me payback? How do I get a smoother run rate? How do I deal with, you know, the spikes? And I think you know, just basic planning of what does your year look like, build a roadmap. I think the spikes, I just mentioned it there, you know, you, you, you have to deal with a 15%, at least a 15% gap in productivity is, is about the normal. So whoever you get is going to be 85% productive. Now, there are other ways of looking at it from a— there are, there are lots of tools that allow you to do things quicker. So actually, you know, it's worth having a view on Can I write code quicker using all the tools that are available? Can I scale elastically both ways by using different parts of the tech stack that, that, that can equate to less people and take the pressure off? There are products that you can buy in, they're already pre-built and you can plug them in that way. So I think these days what's good is there's many, many options that don't— I mean, the start point for us years ago was you, you had to go to the market and hire someone, do a tech test, bring them in, teach them the ways of working and get them going. And that was very much how the banks wanted it to do. And then we, and then we started to enter into the world of building GCCs and building offshore capabilities, um, not always for cost perspectives. Actually, more often not is I just needed more people. I needed to be able to find— and I'll give you an example recently you know, one in a role I had recently where I needed to solve a problem. It was multi-country, I think about 60 countries. I needed about 200 people and I had to get like, there's no way I was going to hire 200 people. I need to go to an integrator and say, can you not get me 200, but I need you to give me some, you know, confidence that there are 200 people so that I can then go back to all these stakeholders, say, right, we're ready to go. So If money is burning a hole in your pocket, that seems like a great problem to have. But be careful what you choose. I think long-term employees I like, but I think having a balance of a team, I mean, in some roles I've been in, we would probably be 70-30. In other roles, it's been 50-50. And, you know, there's an accounting element to this as well as you get to the end of the year and and in some, you know, in some firms there's a bit of a game to play honestly around contractors don't appear on your headcount where you have to declare your headcount just to a regulator, for example. So there's a bit of a benefit to using third parties that way.
Okay.
Yeah, that's where my thoughts go with it. But, you know, I think, would I be surprised to have a board member or an investor saying hire more people, get it done quickly. I think, I think that's absolutely fair, but I think it really is up to, you know, the, the, the leaders in the technology teams to figure out how to get that done in the most efficient way.
Well, headcount, right, either partners or hiring, it's money burn, right? So, so you're gonna be spending. So I think it's going to be the last resource, uh, people will look at initially. They will like, okay, we'll have X people. Can we actually maybe, you know, trim some fat around some processes? You know, do— should we be doing like this? Which then leads to this question is like, how do you, how do you discover where you can actually compress processes? You know, either by automation or potentially just taking away and plugging like a service in. Is there a way you look into this? Is it You know, is it just talking with people and listening for the pain points? Is it, I don't know, putting some data capture so you'll be, you know, becoming data-driven decisions? But in general, if you have a big process and it could be anything, it could be, you know, some cloud management things, you need to move databases, replicas, or whatever, any process which you just want to make it shorter so it needs less hours and less headcount to actually get it done. It's like, is there a way you look into this?
Oh gosh, there's so many aspects, aren't there? I think, yeah, I think my experience has been when it's product related, you know, you sit down with product people and there's a cycle that you go through in planning every year. Certainly in banks, it's always been the same for me. It's almost identical in every company I've been in. But you know, you sit down with product people and the finance people and say, look, you know, this is the cost for the product. You can come down another layer. So if you took a lending journey, for example, you know, you can start looking at what's the cost of acquisition and what are all the components that go into the cost of acquisition. So let's take lending as an example, right? So in banking, it really is this simple. You acquire someone or a company, you do some risk management and some decisioning, and then you do a dispersal. So you've got to kind of look at, you know, and that, and that, interestingly, that model fits for pretty much everything. You just change the language underneath it, right? Acquire, decide, disperse. So, you know, in a lending journey, you would look at, you know, what's all the components to that 50% of it's going to be people. If you look at the total cost, roughly 50%, and then the rest of it is going to be licensing, hardware, software, it's going to be third parties. So actually you can start looking at it, and it's quite binary though, but you can start looking at it going, right, if I reduce my total cost of ownership for that acquisition journey, what does that really mean? There's another side to that, which is why don't I just look at it from a, how do I scale it really quickly? How do I make sure I can do, instead of the same cost for 100 loans, well, acquisitions, originations, can I do 500? And what does that look like? So you can actually start looking at it differently. And that's where I think the automation conversation can start coming in is how do you add friction? Because you do need to do that for risk management and you've got to have enough dwell time for those processes to work. But also If you multiply that across the, you know, the percentage increase, you've got to figure out how that works. But you also want to take friction out. So where is the easy, low-hanging friction that you can take out of that? And there's lots of tools these days that allow you to do that. You know, you can chase down the data flows, you can chase down API calls, you can modernize parts of your tech stack to allow you to, you know, to increase the payloads of data that are coming in and out of the processes. I think there's different ways of looking at it. Cost of ownership, ultimately cost of ownership is what it comes back to. Maybe at the start of your, you know, if you're in a startup, your kind of return on investment is one aspect. But if you're more established, you've got to look at the cost of ownership. And that's where those conversations start. Taking headcount away isn't always the answer. I think if you can also look at how could I do more with the same? As well as how do I do more with less. And the more with less always worries me because that kind of feels like a downsizing, it's not really a growth conversation. So I've, you know, recent experience the last few years around, you know, scaling lending businesses, you've got to really go and look at the data, look at the acquisition journey, look at your decisioning, you know, split into those three. If you can make the dispersals cheap and quick and easy and you can reduce the amount of time that happens, that's great customer. The piece in the middle, the decisioning, that's got lots of friction and it should have lots of friction in there to allow processes to take place that allow you to make sure that you're getting the high-quality decisioning going on or risk assessments. But the start point is very much about, you know, take as much friction out. If you're acquiring a loan application, how quickly can you do that? So automation is really important in the first part. Really important in the end part, but the middle part, you've got to be very careful because if you automate decisioning processes, you might get some surprises when the rules change and they're very much the levers that you pull. So that's a kind of lending journey example of that.
Do you see that potentially changing or improving, Andy, with GenAI? Do you think we at the level where decisions, or let's say low-hanging fruit decisions, can be made.
Yeah, you know, I mean, honestly, it's such a great subject these days because it's probably the most talked about. And I've been on a, you know, number of panels, a number of discussions about AI and banking and financial services. Not new, there's nothing new about it. We've been doing machine learning and we've been using some form of AI in some description for, and I want to say 20 years at least. So, so that's just, that's just really, for me, that's just really clever technology. I think where it's going to, what's changed in the last 12 months for me, without turning this into a Gen AI conversation, but I think there's a lot of, there's a lot of great opportunity for enhancing productivity. And I don't mean removing people out of the system. I mean, allowing people to get to answers quicker so that they can then do a better job and focus more on the other aspects of their work. That's got higher value, but, um, GenAI will, and, and, and it should be used to augment and make life better for everyone if you use it in the right way. So I think it's got its place in some parts, like in the example of a lending or sort of an acquisition journey. Absolutely. It's got, it's got its role to play. Should it be making decisions? I mean, you know what, honestly, the regulators will let us know, you know, the PRA, well, the FCA. In the context of the UK, you know, and, and there are many aspects to this around, you know, decisioning and, um, lineage. And, you know, you get down to the scary bit around hallucinations and what happens if it suddenly starts selling something crazy and you can buy, you know, 500 burgers with, you know, with milk on them instead of cheese. You know, there's all sorts of hallucination stories that we've seen, but I do think Yeah, I'm a big fan of— there are some great tools out there that allow you to get there quicker. I'm not convinced yet that we should be allowing GenAI to make decisioning, but be part of the decisioning process exclusively. We've got a little bit further to go, but no doubt it will come. And I think the research and the conversations I'm having, you know, day to day would indicate that's a couple of years away.
I love that. That's my experience.
I'm looking forward to it. I am. I really look forward to it.
I think it's very exciting. See, see where everything—
for me, it's about consistency. So if you look at small language model, if you build large language models, they're great and give you a wide answer. But, you know, in financial services, we probably want to make that a little bit smaller and get into small language models that feed another small language model that feeds one that then has the decisioning. I think if we try and do it at a sort of at the widest level, I don't think right now that's probably going to have good outcomes for us. And the last thing we need is reputational, um, I think low confidence in Gen AI's use in banking. We need high confidence. So I think waiting and being prudent to wait and being patient is a really important, uh, skill we're going to have to, uh, fend off the excitement of lots of boards and investors saying, you know, what are you doing about Gen AI? And when, you know, the answer is we're all looking at it. We all want it to work, we all see the value, but let's make sure it's not just producing PowerPoint presentations for us with nice graphics. It's got to be a better, it's got to be a much better use case for it.
Yeah. Let's go back to operational efficiency. We drifted a little bit, but it is very interesting topic. So one thing which I wanted to go deeper on what we just talked about, the process optimization through automation and, you know, how you find this. We had a couple of episodes really, you know, and AI came in as, uh, as one of these tools where, where that automation can take place. So it's just an example, but from your experience, what, you know, who actually drives automation, you know, within, say, fintech or other businesses? Is that like a DevOps function? Is that just developers themselves? Is it more like leadership? You need to have some type of like cultural predisposition that everybody should be thinking automation rather than repetitive tasks. How you actually get that done? How can you as a leader be at the distance and actually see that, okay, well, now I'm looking at this process and it seems like a step-by-step copy-paste situation. 6 months later, I'm taking a peek and I can see it's already like a script you know, 6 months later, you know, it's a bundle of scripts. So now it's like a service with the UI which we can give to like interns or something like that. Like, how do you actually see these things happening within the business without, you know, being pushy?
Yeah, I think there's so many facets to that. Um, it's really got me thinking because so if I think about like logical input output here, I think automation, uh, you can look at automation different ways. So if you did a, if you did a risk assessment on a, you know, on a system you know, one of the risks that would come up would be, you know, there's too many manual touchpoints or there's too many manual steps that cause outage or cause problems or, you know, and the list of problems is huge. So I think you can look at it from that perspective is if you go back to good old-fashioned risk management is, you know, what have you got there? What could go wrong? And if the answer to what could go wrong could be solved by automation, it's, it's a good thing to go and do. You need to think a bit wider about where could that go wrong. And, you know, you don't, you don't want to wake up one day and find out that you've used your entire year's budget on scaling something that someone else just hit a button for. So I think you got— I would, I would always look at it from a where does automation play the biggest part of the value adds for different use cases. So is it going to be better for customers? Is it going to be better for cost of ownership, is it going to be better from a risk management perspective? I think maybe if you looked at those three aspects, I think that would get you going in the right direction. In terms of where, where would it sit? I've seen automation teams sit separately in, in the, the sort of IT, you know, shop. But my challenge with that is often been, and, and I've had a couple of examples where I like having an automation team come in and maybe try and take some friction out of a lending journey, as an example that I used earlier. But the problem is, is those people can come in, and you've got to give them access, and you've got to give them permissions. And that's where it gets a little bit difficult. And it's the same with DevOps. If he's like, DevOps is a job, or that, I prefer to think of DevOps as a culture. But you know, the whole notion of DevOps is about continuous integration, continuous delivery, there's lots of automation in there. So I think the, the answer to that part is what tooling can you bring in that allow you to build scripts or to sort of build the pipeline and automate and scale or build containers and allows you to do that without you being involved. Now, one thing I love about DevOps is like, you know, and there's a great example of VIVE here where, you know, we're building, we're building a bank is almost entirely infrastructure as code. And I wanted to go a step further. And so you could call that automation because fundamentally it was automation. It was, it was infrastructure as code, compliance as code. There were elements to it where I could build an environment and, and just, and take it down each night and build it every morning on demand, elastic, truly elastic, and get going. And then I could ship it to a different region. You know, if I was in, you know, Europe, I could ship it somewhere else within reason. And obviously, there's a data piece there that you got to think about. But I try and think of it that way. So I've seen, I've seen teams in different places. And, you know, DevOps is another challenging one, really. So you have a group of people who have a lot of access, and trying to manage that and do the automation. Back to what we were talking about earlier. Yeah, you can— automation is great. But and you can, it can allow you to go very quickly. But sometimes going quickly can cause you a lot of problems. Like it's another thing that's lovely about, you know, about cloud is cloud is great because it allows you to do things really quickly. It's also bad because it allows you to do things quickly. And in the world of financial services where you need to have, you know, very predictable outcomes, you need to be really clear on what you're doing and why you're doing it and have an answer so you don't wake up one day with holes, with leaks, with disclosure problems, with compliance problems. So Automation has its place. But I think I, you know, so look, here's a different perspective is automation should be like in the same way risk management is part of everyone's job, or thinking about automation is also part of everyone's job. How much you invest and how much you want to do is not easy to articulate.
So it's almost like if we—
way of life.
Yeah. And that's what I was going, it's almost like a mindset, like maybe should be part of your talent acquisition and hiring interview type of process where you make sure that you hire people with that mindset, what they look at.
But if you think of it, if you go back years ago, you know, sat down there, you know, writing COBOL and now, now, you know, a lot of code will write itself or it'll correct itself or we've got tools that allow you to sort of, you know, we've developed tools where I work now, where you can look at— it could summarize the code for you as you're going.
Yeah, I do agree. I mean, coding right now, it became—
I don't want to say easier because I don't want to offend your audience, but there are— yeah, it was all harder in my day. But there was— it's higher quality these days because we're asking systems to do a lot more than they did you know, many years ago. And if you think, how do you get, how do you get all that data from a backend COBOL mainframe core banking system into a mobile app? And that mobile app does everything that your browser used to do and some more. You know, we're asking people to code a lot, but automation is a mindset. I think in the same way security should be your mindset, in the same way resilience and compliance should be in your mindset. I don't think anyone comes in every day— I don't know anyone that would come in and go, I'm going to write bad code today. They come in, but we've got tools that help them, help them do it.
Yep. Um, so two things which I would like— one thing which you mentioned is infrastructure as code. We all— I want to say all the listeners likely knows that it's a very common way of, of building clouds. We mentioned as well compliance as code, which I feel like, you know, our listeners might not be aware of that, especially if they are in regulated industries like fintech or, you know, online gambling or health tech. Can you give us a little bit more insights? I mean, what is that? How you guys did compliance as code?
Yeah, so I think, you know, really it's— I suppose it's an extension of infrastructure as code to a degree where you're actually trying to bring in those elements of compliance and GRC. So, you know, governance, risk, and compliance and security and making sure, you know, when you're, if you're shipping code through pipelines, you know, have you got something having a look at that to make sure that that's not going to do something somewhere else, right? And how does that integrate? How do you make sure that you're not building APIs that are pulling in data that doesn't have consent? How do you, yeah, there's a load of facets if you think about it. Compliance as code is quite a broad statement, but building infrastructure that is compliant with cloud, with, let's talk about, so in the FCA handbook, they've got the CISC 8 is the Systems and Control. Now in there's a load of requirements that you have to comply with. And so, you know, compliance and code is maybe looking at that aspect as well as how do you make sure that your applications have a sense of what is legally or regulatory compliant. And there are lots of rule books. I think if you get into, you know, certainly if you get into cap markets, there's lots that you have to do and you have to do it in the right way because you have you have sort of like you have product compliance, product compliance, then you have market compliance. So you've got two different views that you need to be able to manage there. So that's a big part of, I think, the future of just building out compliance as code and having that mindset of how do I make sure that what I'm doing is the right thing, not just from a security perspective, but you've got to start thinking compliance. And there are new regulations these days, you know, with critical third parties, with DORA, with Basel. There's numerous newer regulations and newer oversight and scrutiny on technology, more now than there has been at any point in the past, where even now, your third parties are being— it will come under more scrutiny for things like DORA, the sort of Digital Resilience Act. So I think compliance as code is definitely something we're gonna have to start thinking about. How do we do this, particularly around data? You know, GDPR is not a new thing, but it's still, you know, trying to understand how how to manage everything is still, um, still a challenge for everyone really, because data is the lifeblood of banking. Um, be difficult to say it isn't. Yeah.
Okay. Um, you mentioned speed being potentially a course of disaster if you're going too quickly, you know, leakages, holes, and things like that. Fully agree. Uh, I think there's a really really a strong notion of the fact that you could go too fast and, and, and, you know, jump over quality bars, you know, jump over details just to get things out, you know, hit the milestones, hit whatever commercials people looking for, and then as a result actually create enough product tech debt or whatever debt which then comes back and bites your ass. Outside of that, another drawback which I see with employing BigSpeeds is actually that people, they start talking and saying, look, I have no time to be creative, I have no time to actually build real innovative things because my mind is constantly switching between these priorities and I'm just always doing and doing and doing. For you as a business which is competing for the market share, you know, you, you want to be creative, you want to be creating new things, You want to be creating new things for the customers and the market, so you eventually develop some type of moats or whatnot. What's your take on this end? Is it possible to be like— let me rephrase this. How would you, how you did in all other companies to make sure that the creativity and innovation is, you know, still there? There's still time to do things which don't exist yet?
Yeah, I think a lot of the time, so in banks, there's just, you know, for me, it's always been— there's been this sort of segregation of duty. So the world of sort of innovations and labs and studios, they, they have their place within the organization, but they, but they sit in environments that aren't in run, they're not really in change, they're definitely not in security. So, and I always think of, you know, banks as, you know, run, change, secure, govern, right? Those four pillars, they kind of sit outside of that. And I think that's a good thing. It allows them to be creative, create those sandboxes, create these environments where they can kind break things, connect into the FCA sandboxes to test their thinking. There's lots more of that around these days. So actually, look, if there is a requirement to be, you know, really speedy and you want to take risks, then you will. And if you want to put products in the marketplace that aren't quite ready, and, you know, there are many companies that have done that and done really successful by using the consumer base to tell them what's wrong with it. But there are, you know, there's a place for that. But we work in financial services in a regulated environment, so you can't really do that. And reputations— and we've seen some good examples recently where you've got well-established products, there's been some enhancements, and it's managed to cause a lot of problems. But for me, getting around it is, you know, innovation is really important. For me, innovation is more about business process than tech. So, you know, get your teams focused on what's the, you know, work from the customer problem to the business problem to the technology solution and think of it that way. But it has its place. I like having teams that have time to, to be innovative, to think about what's new, to, to, because it is inspiring for your day-to-day as well, to not just be sort of going through the grind of here's all the backlog. You do need to have an eye on the future because like we're talking about change early and the change is really important. So innovation has its place. I do think it's good to have segregation of duties so you make sure those people don't have access to do something and put something live that just should not be there. And the last thing you need is reputational damage because, you know, and I've been through many, many really difficult situations with systems and outages that just, they just cause problems for years and years and years, and the reputation is really hard to get back. So Yeah, my advice would be don't stifle it, but just make sure it's controlled in a way that you can set people off in a direction that's got a clear outcome that's going to benefit someone, or you're— or, or they're trying to do something that no one else is doing, but, you know, put some controls around it. So I could get back to risk management, really, and operational excellence is don't break, don't break what's already working elsewhere.
I love that.
I thought— so one thing, so a bit of— I probably said it to so many people they could quote it back, but I've always said to you, yeah, don't do something you can't undo. There's something to, uh, to start thinking about.
Yeah, it's, um, I think that that's the notion with the two doors of the Bezos analogy, is like one door is that if you can undo things, just go ahead and do it, you know, like the risk is manageable. But don't— yeah, don't do things you cannot undo. Like, the only, the only thing you're left is actually move forward, you know. Don't—
I mean, we've got databases, like, infrastructure as code is a great one, you know. We sort of, you know, and things like kind of looking at Chaos Monkey, and there's loads of great stuff that we've tried. I've tried to do over the years, encourage the teams to say, you know, let's try and inject an API somewhere and break something and see how we recover. Let's see how resilient we are. And you know, we got a bit too brave and things fell apart. But, you know, do it early and make sure you know why you're doing it, if it's for a good outcome. And, you know, and I think the other thing is, you know, certainly done some resilience work over the year where we've done failovers or we've done DR, disaster recovery. And, you know, it's part of your license requirement. You have to do it, you have to prove it. And those rules are getting much, much stricter now. Is tell everyone you're doing it. Like, the last thing you need is is the chairman of your company ringing up and saying, oh, I've just gone on the website, it's not working, because he didn't get told that we're doing it. And I think, you know, get back to the communication. But yeah, there's the— I think, you know, I've always encouraged teams, you know, many, many over the years where, you know, team members just ring up or they walk over and go, oh, I've just done something I wish I hadn't done. And, you know, you have to get your arms around them and say, look, you know what, let's, let's just, let's solve, let's recover first, uh, because everyone's gonna be asking questions why, but let's, let's, let's get our arms around this quickly, figure out what we've done, figure out if we can, you know, roll it back, and then we'll ask some questions about what, why, why that may have happened.
This is actually very nicely leads to, to your question, Andy, how I've seen leaderships, uh, who take a crisis and, and convert it into opportunities And, you know, giving you an example, something happened like you just said, someone does a mistake. Let's say the production is down and, you know, after that they do retro, they create initiatives, they find the holes either in the process or within the product, they fix it and it never happens again. For me, that's a crisis converted to opportunity and then just bettered the whole business. And other leaderships where they have this crisis, they deal with it. And they then back to product development, back to pushing something out because there's a deadline. So effectively, and they along the way, later down the line, they just end up firefighting all the time and they're almost like, okay, well now we need to create like a defect team who just going to shield us from product development because otherwise we're all distracted or something like that. So what you just mentioned for me sounded like, you know, like a crisis to opportunity situation. But in your take, um, can you share, um, you know, a crisis which, which happened and, you know, what you as a business did as a team and, you know, what happened after that?
Yeah, I mean, I've got a few here. So, you know, at their, at their most simplest level, you know, a, you know, a fire on a, a fire on a floor and everything gets burnt. And, uh, you know, I think as a part of a UPS test, you know, is one example where where the UPS, Uninterruptible Power Supply, test was done, fried all the PCs on the training floor. So they get the team in at the weekend and change all the power packs on them, and then we were back up running on Monday morning. And yeah, I always thought my role in that was to sort of, you know, help shape the conversation, communicate what was going on, and give people a level of confidence that we'll be back up and running. So it's kind of the lowest level, and yeah, very calmly said to the ex-co on a Monday morning, yeah, we're all back and, you know, deadpan, yep, we're all good. Because actually, you know what, it's not a big deal. It's, you know, we had, we had it under control. I think there are other extremes where, you know, through no fault of our own, you know, a, a situation where we lost our storage area network through a flood. And that flood shouldn't have happened, but it did. And we could have avoided it with a you know, $2 umbrella, but we didn't. And some water leaked into a road that then dripped down and dropped at the lowest point, and that was— the sand was underneath it. So at that point, and I remember the time of day, and, you know, at that point you're recovering a situation that is probably quite straightforward. The recovery actions that subsequently happened after that was to open up payment gateways, and sort of they were in a mirrored environment. So you know, we closed down one side, replicate the data, send the payments out, but we'd never tested it and so we opened up the other side. That then replicated out and the duplicate payments went out again. So at that point, the situation with the flood was under control and then we went from in not a great shape because we'd lost, you know, a big part of our environment to an even worse situation where we've had to shut everything down. So What came out the back of that, and, you know, and lots of pain to go through trying to kind of recover the money and, you know, counterparties and that sort of thing. So you're into crisis management, risk management, and as we're on a call for 36 hours, all quite painful and a good memory. But what came out the back of that was clearly we didn't understand our architecture for resilience. Clearly we'd never done DR before. So at that point, yeah, I was in a role where I was quite happy to take the take the lead on, let's build a payments team and let's build an environment and architect an environment and build the system to make sure that it had resilience. And we went from having never met an SLA to having never been below 100% because that scenario pushed us and it was really challenging. And I think The reason it was just a real learning mindset in the team, and again, back to automation, you know, at the learning, the sort of learning mindset was about how do we make sure that this can't happen? Not— and that's where we put our effort in, rather than the start point was like, you know, how, why did it happen? Because it's very, it's very easy when things go wrong to start, you know, well, why did it happen? You start putting a lot of energy into trying to answer the question. We just sort of took us a few steps further and said, right, what do we do differently? Like, let's architect this so it can never— and think about all the different scenarios. So you're into sort of extreme plausible scenarios, because we'd have never come up with what would happen if the data center flooded. That wasn't— that was— that didn't exist on a risk register. So I kind of moved us past that bit, right? Let's go extreme plausible. So that's one example. Of just learning. But I think, you know, there are other kind of smaller ones where you just have situations that, that present themselves. Things break, you know, features get posted out, you get, you know, um, you get, uh, behavior in systems upstream or downstream that you have no control over, and you just have to manage with what you've got. And I think you post those events is just a matter of making sure that you've got a risk management framework that you can then sort of log these things and go and talk to the teams that are involved, security, risk teams, certainly in technology and big, certainly bigger banks. You have, there's a lot of people that do risk management. You can go along to them and pull them in. So I think crisis to opportunity, again, mindset, if you can think about that, certainly from the, you know, my time in the military as well, is you're always thinking about what could go wrong. Because the impacts are huge. And I think in financial services, when you're dealing with a massive trading floor, or in a private bank, or in a sort of massive commercial business where you're, where you're, you know, you're shifting a few trillion dollars a day of accounting flows, small issues need to be, um, you need to deal with them really quickly, take a deep breath, get through it, and then get on to the sort of how do I stop this happening again. And that, that for me is the opportunity of of being able to go to a board or whatever it is, is be able to go along and say, here's my plan, here's my idea to make sure this doesn't happen again. And because the cost to it is X, and you can normally do that quite easily with a good finance person will help you generate that number.
I love that, creating that theory of opportunity, opportunity itself, then looking for stakeholders by it to actually execute that by default is opportunity. Okay, I think that's what we're finishing today, Andy, and I really loved your clear roles, clear goals, and clear accountability piece. I'm going to steal that. You're probably going to see a post on socials.
I can create it. It's a good mantra. It seems to work quite well.
Yeah, I think a lot when it comes to operational excellence that we talked today, you know, stems from that. You know, it's, uh, it's very mindset-driven. You know, we touched on automations and, and, uh, change management and, you know, risk management. Or all of these are, as much as you want to make them actionable, you know, they are very broad topics. And first you want to make sure these exist in your culture and in your vision as a business. That, you know, you're hiring people towards it or you're educating, growing people towards it. So that just becomes, you know, a natural order of things within your business. And hopefully, you know, the operational excellence just stems out of that naturally. So if people have more questions about, you know, the topic and the— how can they get in touch with you?
I'm on LinkedIn, Andrew Parsons. You go search me up on LinkedIn and happy to answer any questions. I think, you know, a lot of it is about sharing those leadership stories, sharing those leadership insights. I've got a massive amount of experience in lots of different areas and sectors and worked around the world. I worked in lots of different types of businesses. So yeah, LinkedIn is, LinkedIn is great because I can respond to that during the day. It's not, it's not too difficult. But it's been great talking to you. Huge amount of subjects to cover, it always is. But I think, you know, it's been really, really interesting and tested me a little bit to have a tap back into some of those kind of horror stories and go, oh, let's see, you know, that was interesting. Interesting at the time, it's probably horrible, but in hindsight, it's, uh, it's nice to be able to kind of reflect back and think that we did the right thing. But, uh, thank you.
Thank you, Andy. There's no failure, it's always just a lesson which you'll learn. So, uh, to our listeners, uh, just scroll down if you're watching this on YouTube or you're listening as a podcast, in the show notes you'll see the details for Andy. Click through that and have a conversation with Andy as well. If you want, leave us a review in Spotify, you can leave a comment in CastBox, you can leave a comment in Apple Podcast. I'm really not sure, but I think you need to leave review with the comment. Leave a question there, we'll pass it to Andy, we'll come back to you with an answer. And as always, until the next time, stay safe and see you online.
More from Ben
How to Validate an AI SaaS Idea by Building It in a Weekend — with Richardson Dackam