Software Estimates in the Age of AI | Developer AI | CodeNinja Podcast

Wrong estimations mean that you're going to be either overcharging the customer or undercharging the customer. If you're trying to overcharge, you might lose the deal. If you're not, you know, if the, if the deal is not lost later down the road, it will cause frustration for that company. If you're undercharging, then obviously that project is not going to be commercially, you know, viable for UT because of this friction between teams doing the estimation and teams doing the execution. There's this sense of not owning the estimates and then there's a blame, etc. As well. Repeat these cycles over and over again. There's a lot of baggage. People, you know, go through these experiences. It has an emotional tone. Software estimations. Why 80% of the software estimations are wrong, why they're wrong, why they're wrong. It's a one trillion dollar industry when it comes to software services and from last 40 years, why they couldn't figure it out, how to give the estimates. In a way 80, 90% closed. But I mean today software estimations look like that. You need to structure something, you need to have a mental model in actual way. It has nothing to do but how you plan it, why it is the way it is. I think when I think about software estimations and you know, the question that you ask clearly what we need to go back and see is what are we trying to estimate the software estimations? Essentially it's the uncertainty and we're trying to predict the future. But then the question is, how are we trying to do it? And when you go back and look at most of the case studies and mostly the prevalent practices, is essentially subjective estimations. It's humans who are trying to predict the future based on their intuitions, some semblance of data, their own biases and how they look at uncertainty. You know, even, even if you look at waterfall or AGM practices, you know, we talk about story points and so on, so forth, but things change. There are so many other forces that come in. It's a dynamic organism. You know, when you get into software development, things change. There are uncertainties, there are complexities. When you anchor software estimations, everything that now at the start of a project, everything that you're not trying to do is in relation to what was anchored. And nobody thinks of software estimation as a dynamic organism which has to change and adapt and move dynamically as the uncertainty and complexities around the whole thing changes and as well as, as new information comes in. So what I'm Trying to get to is that we need to develop a completely different perspective on estimations and it should not be something that is just done right there and this, you know, cast in stone and then we forget about it and then we move forward without looking at, on how, you know, the landscape changing. Now that is a bit of a, you know, so it's interesting because when we look at services industries, for services industries, it's a completely, it has a completely different significance because that's where it all starts as compared to you if you're trying to develop your own product and you can be more reactive and so on, so forth. In which case even agile approaches give you ability to react, not necessarily estimate better, you know, but so, so when I come to service industry that it's, you know, the way I like to think it of it is that it's that commercial linchpin which, on which your relationship with your stakeholders, clients, customers based on. So that's where it starts, you know, that's what you want to ask the customer. You spoke about uncertainty. I have been part of those workshops and discoveries which went on for years, for eight months, for 10 months and where people try to manage and write everything that could go wrong and try to bring it into your documentation so they know and plan accordingly. But what happens discovery workshops, uncertainty. I mean, I just want to make a point here. So for example, when SpaceX started, they started off is like what the hell is cost plus contract? This like because you, because these, these systems are so complex you can't really come up with a cost. So what that is is whatever the cost plus contract. So you would charge a margin on top of it. SpaceX came in and they said no, that's wrong. It has to have a fixed cost of this thing. If this thing is possible for system as complex as a rocket, why software is so hard? Companies, companies like accenture, they booked $2.6 billion or something in Gen AI. I mean but they couldn't come up with a way software requirement capturing. For example, whenever you want to work with these services businesses, they ask for discovery workshops that go for six, eight months. So why innovation cease to exist? Rocket engineering requirements. But software, why is it so. It's an interesting one. It's a two part so we can focus on how it's done. You know all of the examples where you go into these longer work, you know, workshops and try and estimate this. The other thing is where are we when it comes to now, how the world has changed, where we're sitting at and how Why? I think this is a problem ready to be solved. It's a matter of, you know, just getting your head down and doing it. So I, I genuinely don't think it's a problem that can't be solved anymore. We'll be sitting down. But historically what has happened. Let's just talk a little about that. There are many factors in place if you're into a six month discovery workshop, right? There are cases, for example, I'll give examples. There are cases. The team that has doing the discovery is not the team then developing it. So what really? So there could be different team doing the discoveries and then there could be different team actually building it. But essentially what you're doing is in that case is you're taking a gamble with the uncertainty by different set of people doing something and different set of people doing something because there's no objective truth here. Unless we come up with a mechanism of defining that objective truth, then that's really what is happening. Is the team who done that six months of work, all of their biases, all of the things that things that are going to change when that piece come, goes into execution, is lost, is not accounted for. Requirements is one part of the whole equation. Okay, Having an understanding. But when a team comes together, they, in a way they come with all their expertise and you got to benchmark those teams, etc. As well. That's what becomes that organism which is going to deliver that piece of software or piece of code and so on and so forth, which could be significantly different from at the time when you build. You know, you did that estimation or you did, you understood the requirements. There could be nuanced cultural things as well. And I want to actually talk about what happens with this estimation. What kind of impacts you see having not accurate estimations because of the complexities and because of the teams and people. And it's a human method, it's an expert method, right? Even with story pointing, a bunch of people sit here and they, you know, try and come up with this ass and there's always this, we call it the, you know, the positive positivity bias. You know, I was in a meeting the other day with one of the, you know, a CEO of a company and he was telling me that they try and do these ground up estimations, forecasting basically. But what really ends up happening is that, you know, at a granular level, whoever is there, you know, the person who is supervising, when he asks them, you know, the person down the hierarchy for estimations, they're very gung ho. About it, they're very optimistic about it. So the optimism bias goes all through the process. So people want to do it. People would say that you will get it done, hence the estimates become unrealistic. There are also cases, the cases where people have a long standing history and baggage of not being able to deliver projects, they would all overestimate the projects as well. And hence especially for services industry it becomes a big issue because then you're not able to sell because you, you would know that most of where you lose the deal is essentially commercials and you know, apart from all the other things. And that's where you know, overestimation is a big, big problem in software services industry, especially with, you know, the way AI has come about. So you see all of these challenges coming into, during the estimation process. Now there are ways to solve this and really where we're sitting on right now I believe is that machine learning and AI, I believe can actually help us solve this problem problem because now we just don't have models, machine learning models, you know, the traditional regressions or time series, we just ingest data. Now we have attention based transformer models who can understand language as well. And when you combine the two then you add all the context, all the complexity of what a project possibly could be, you know, all the way from requirements to the sentiments of the teams, you know. So I'm talking about retrospectives that you do, the status reports that are published that you know, issues that you see across the historical projects that you've delivered, documented in, you know, maybe in a language, you know, in form of language reports, requirements, so on and so forth. Even things like, you know, these models can go and look and see that, you know, front end to back end ratio, for example of a particular project. Integration challenges. I don't want to. So before we go in fixing and I bring in the other admins, if I ask you based on your experience, give me three major reasons why a software architect or project manager fails to estimate properly and where they go wrong. So what is number one reason why a project manager fails to estimate and fails to coping? All that uncertainty. Why, why it happens. What is one thing? If you have, if someone, if someone is trying to fix that, what is that one thing a project manager can do to fix that thing given the scope of the project is an average, average scope. It's not a hugely complex 10 year software development. The one thing that a project manager or an architect can do is look at so some kind of historical data to, to be able to do it. It's all based on intuition right now, the way it happens. So you're saying the entire software estimation is primarily based on intuition? It's intuition, it's. It's also normally all the methods that even, you know, methods like part and all of these, you do, you know, people sit down and they throw a number based on their own individual biases and intuitions and whether you take an average case or worst case, you know, methodologies like part etc, it's just people sitting and just throwing out a number based on their experience. It could even get affected on that day when that estimation was done because how somebody was feeling about something. Have you ever come across any soft estimation based even widely different estimates? Most, most of the times that that happens. So if there's no ground truth, you know. Yeah, I would would really like to have a study where estimations are done at different time of the day. Even if it's a same person. I'm pretty confident that, you know, that might change. You're more focused, sharp, your brain chemistry different during the day, in the evening, it's completely different at night. So I'm pretty sure the study will say, you know, you do it in the evening and it's probably different. This is all your intuition, expert knowledge coming in, why someone didn't fix it. It's been 40, 50 years now, especially in software, you know, why it's not fixed. So historically we can improve it, you know, using like I said, machine learning models. Like, you know, you can have the data and all that, but what you don't have is all the other complexities or the other contextual things that might have happened to the course of that project. And that's where this machine learning, you know, that's where the AI, you know, the attention based things come in, where these models can actually go through that data and learn what really happened during these projects and then make better predictions. So for example, you would have planned versus actual data in project, right? This was planned, this was. And you can feed it to a machine learning model and it can do some sort of trend analysis. But what you didn't have was a model which would actually go in just all the requirements and determine the complexity of that requirements as well. And which could go through the historical data of status reports which were just escalation reports, steering committee reports, what kind of issues were highlighted, that context was missing. We didn't have the technology to be able to do that, that large language models can do that. Now when you combine the ability of large language models as well as Traditional machine learning techniques. I think that's where the solution at the intersection of its which I think is the solution is this thing out. You said it's a huge problem by now. We'll figure it out. It's a huge problem. I mean intuition based estimation, let's call it this, not like AI with a separate question like do you think that if AI able to give us the estimates we would trust on that? It's an interesting one. So the way I think about this is and especially you know I was thinking about this from a problem in. And in most cases that's what I see. You know, you look around even projects that we do and some of the other companies might work with. It's all, it's most of the cases a different team doing estimations and different doing execution. If AI is doing it, one of the key cultural impacts it has is there's no resentment amongst the team which is going to develop. So they won't blame anybody for these estimations. What they have is a central alignment truth sort of truth which has not been imposed on either party. That's a baseline they take and then they work around it so they could improve it. They could see that what are the shortcoming and so on so forth around those estimates. I think that's a value. There's a very key value that I think we can all tend to ignore. I want to talk a little about what the ripples of this estimation is by the way. I think very important to stand. I don't think we really think about these things wrong Estimations mean that you're going to be either overcharging the customer or undercharging the customer. If you're trying to overcharge, you might lose a deal. If you're not, you know, if the, if the deal is not lost later down the road it will cause frustration for that company. If you're undercharging then obviously that project is not going to be commercially, you know, viable for eat because of this friction between teams doing the estimation and teams doing the execution. There's this sense of not owning the estimates and there's a blame etc. As well. Repeat these cycles over and over again. There's a lot of baggage. People go through these experiences. It has an emotional tone, it creates trust deficit between the stakeholders and the developers as well. It also promotes a culture where all the time you're trying to explain the deviation between the plan and the actual and somehow that not being the developer's fault and you go through a bunch of experiences like that and you pretty much destroyed the culture and know it's not a fun experience going through this over and over again stemming from this one idea. So I just want to lay it out on the table. It's as deep and pretty much affects the company pretty much the life cycle of every step of the way through the project and so on and so forth. Now again wanting to solve it, I think can AI solve this? I think we have the tools as I explained, combining traditional machine learning statistical techniques with large language models and it's a bit of a data problem as well. Right. My question is like for example, if I figured out this is going to take 5,000 man hours, let's say AI, our model says that this all work, it's going to take 5,000 man hours. Are we going to trust that number? Okay, yes. Because if we, if we do it means that whatever happens during the project life cycle we would say, I mean that's what it is. We are, I mean we can't go back and say because both macro, whatever are we going to trust the. Because that's, I mean we'll build that model, we would have that data. But if we build that model, stakeholders, clients, over a period of time when all of this process improves. Obviously Banjayga but I would rather go from approach where people have a consensus without resentment to take a number and work with it. Like I said, if somebody is doing estimations there's a blame finger pointing at some point. But if you're using leveraging on AI and obviously now we are reasoning more model now you can look at chain of thought as well, you know, all of this thing, you have certain confidence and it's an ongoing improvement journey as well. If the, the other way to think about it is also if we have any confidence in what this technology would be in the next couple of years, any rate of improvement that we can think of then starting and embarking on this journey with what we've got and, and then building on top of it is the way to go. So maybe to start with you've got a number from the AI model and then you've done some subjective estimation as well and then you're, you know, trying to look at both and then maybe that's how you start. And as you, you know, as more data comes in, as more, you know, things come in, it keeps on getting improved, so on and so forth. It is better to start on a, on a different note, one is estimations, that's one thing. But one of the things One of the challenge that we have is senior lot people who are 10 plus years of experience estimates, but they do not factor in this massive, incredible engineering velocity that comes with using AI tools, for example, Copilot. And they don't get it. I mean one of the challenges, they don't get it. I mean, I mean, is it a good idea to convince them that this is a modern way of work? Because they say no, that's it. And if they're not, it's not being reflected in estimations. I mean clearly you, you cannot even compete, you can't deliver that value to the customer. Yeah, I think I'm pretty clear about this. And one of the reasons why, by the way, one of the. And you didn't work, you're doing, you're using cursor and all, all that stuff. Yeah, yeah. You know, what value does that? Do you want to talk about that experience? I mean, I want, I want to. You've been using cursor. You were. I had not written a line of code in 15 years. 15 years. And now, and then I just. You done it in three hours. Yeah. And I, I just built a, you know, an AI agent is what I'm building. And it's just unbelievable. You know, somebody who's not written line of code in like 15 years and then I just start off and it just, right away, you know, the kind of progress that I made in two, three hours is I was shocked, you know, I had like goosebumps, you know, what was going on, being so close to the technology. But I want to, I want to highlight a key point here. Now this has become non negotiable to me with what I've seen. Experience is non negotiable. The reason why you need the technique also is, and I know for a fact that a lot of companies are struggling as we speak about this because ignoring, you know, code generation capabilities of these, you know, generative models means that you're not even in software services, you're not competitive anymore because your estimation is going to be way too high than you know, what people are willing to pay. Now if you don't have an estimation based on these techniques, the problem you face is you can't incorporate the value, you know, the co generation can bring in. Why? Because when you try and get a good estimate that is probably from a developer who has 10 years of experience or more so on so forth, and that intuition and that bias is already there, but you don't have that data that what percentage of improvement is the code generation tools going to bring in so nobody has intuition about it. So if you don't have a subjective way of measuring the data and you know, instead of expert judgment having a more subjective way of doing it, a more mathematical way of doing it, then how are you going to know what to do? Because traditional method of intuition is outdated. Not because of anything, just because of the fact that some tools have come about who can generate code anyway. So whether it's 10%, 20%, 30%, 40%, you don't know and I don't know how long is it going to take for people to start using these tools and develop that intuition? Intuition is probably going to take a long time. If you start collecting the data it'll probably be a lot easier. Maybe in the three to six months you'll probably have the data and you know, these models will know. Okay, this is the efficiency you can bring about. So that's another think existential question. We're talking a lot about AI, AI and software estimations or engineering velocity. These new co pilots and cursors and all that. Facebook is saying they are probably slashing mid level engineers Salesforce they're not looking to onboard any new engineer because they have just found this so incredible engine engineering velocity. They're looking for hiring sales people taking no more engineers. Your graph U.S. government labor status software jobs. That was like starting from 2022 onwards, Someone who's been in the industry from last four or five years doing job. What do you think? How, how should they think about this, this, all this what's happening. So because these companies are focusing on either elite status engineers strongly well the Andre Karapathi types or entry level young engineers and giving them those tools and saying there's no entry level engineering. How do you say that? How do you see mid level where they should focus on I think where the technology is right now and obviously you've seen the trajectory. I mean I, what people are not really thinking about is you know the reason why I say that. And very recently I had the chance to you know, for one of our senior positions with the customer interviewed more than 20, 30 people around different organizations, so on so forth in leadership roles and what people are really not thinking about is where the trajectory is. You know just to put some things in perspective. You know Mark Zuckerberg said okay, we're gonna have by the end of 2025amid level AI agent which could code a mid level engineer but it's going to be very compute heavy. Right? So yeah, I mean we're thinking all right it's compute heavy. Maybe it's not a, you know, like it's one of those quantum computing things where it's just not practical. In last three years when it comes to efficiency and Quantum cost reductions. 1200x improvement since GPT3 to GPT4001 so has happened already 1200x so we talk about Moore's law and it's gonna, you know, every nine months you know the cost gonna be reduced and you know, you but this is unlike anything that has happened 1200x. So we might be talking about this but in a couple of years time if the rate of change is anywhere close to what it is, you know, in we don't even know in how much time the cost compute happiness is going to, you know, come down so dramatically. Especially with the you know, reasoning model models coming in which are extremely good at coding and as they improve again looking at the trajectory. So I think now these tools are superpower for developers. They need to have different deep develop deep familiarity, deep knowledge of using these tools to start with so that you know, they can improve their product. That's how you compete. Right? For now that you know, I could be way, way more productive if I'm leveraging on those tools. You know, if somebody like me after 15 years can actually, you know, build a piece of software in a day leveraging on something like cursory. I'm pretty sure people who are experienced can do wonders already with the using these tools and improving the productivity and so on and so forth. I really, really believe that. And then there's a larger discussion around building, you know, essentially expertise around domain and you know, deep domain knowledge and so on and so forth and that's there. I think right now everyone's got deep familiarity with the tools and deep domain knowledge. They if we have both of these things probably gonna survive this. And let me tell you, 80 to 85% of software engineers, it looks very absurd but 80 to 85% of software engineers have not have any familiarity with the tools they've been doing. And look how odd it is how software engineer is not familiar. But that's what it is. For example, if you are working on one stack, deep familiarity, your ability to look at how many different technology and tool sets is available and trying to use. But that requires a new learning curve. I'm telling you based on my experience and talking so many people, 80% of the people don't have any familiarity in the tools. And people use their deep domain expertise sometimes to say that's the reason they don't want to have, they don't want, they don't need to have that deed similarity because they're so good in domains. They are here not for engineering but for the domain knowledge and domain and I think that's not gonna work anymore. You need to have a domain knowledge and you need to have, I mean if you're a software architect for let's say for a logistics industry, yes, many expect. But it does not mean you can have logistics. So now your executive care for software architect. No, you, you need to know what exactly? I think so it's a, it's a, it's a red flag for you know, countries like us, etc as well at least having that sense of urgency. So basically what you're talking about is there's no, there's not even a sense of urgency. I mean nobody's saying that, you know, next week or next month, you know, suddenly developers allowed it. But you got to look at where this is heading and at the pace. All I would say is just try and look at where this is heading and whatever rate of change right now you're seeing even discounted it by whatever factor factor that you want to and then try and see where this could be.