Amazon CEO's LEAKED Conversation Reveals Stunning Truth About The Future Of Software Engineering

So software development, AI and the job market is rapidly evolving, and in a recent leaked recording, an Amazon Cloud chief tells employees that most developers could stop coding as soon as AI takes over. Now I know what most of you guys are initially thinking. You're probably thinking that this is an AI hype video where I'm saying that oh no, all AI software engineers are going to replace all traditional software engineers. That's not what I'm saying. I'm going to break down this article and show you the actual, real grounded truth that's based on actual industry wide developments in AI community that most people aren't paying attention to. So this is an article from Business Insider. I find it quite insightful because this is something that actually happened in a leaked conversation and with the rate of current AI developments, I'm not sure that this person is entirely wrong, although I think there are a few nuances that maybe this article doesn't pay attention to. So a summary of this article basically says that the Amazon Web Services CEO Matt Garmin shared thoughts on AI during an internal fireside chat in June, and Business Insider obtained a recording of the meeting. Garmin's comments were a kind of advisory nudge rather than a dire warning to software engineers. Hence the part where I'm stating that this isn't a kind of oh no, all software engineers are gone. But like I said before, AI is most certainly going to be changing the dynamic. So let's take a look at this and see exactly what was said and what this truly does mean and and any other further things for the industry. So he says here the software engineers may need to develop other skills as soon as artificial intelligence takes over many coding tasks. If you aren't familiar with the current concept of how good AI is, many people have been speculating that AI is going to replace many coding tasks due to its ability to perform many coding tasks in natural language with a remarkable level of efficiency. Now, I do think that sometime in the future this is going to happen, but there is a bit more detail that you do need to pay attention. And it says that's according to Amazon Web Services CEO Matt Garman, who shared his thoughts on the topic during an internal fireside chat held in June, according to a recording of the meeting obtained by Business Insider. Now here's where he gives his prediction for the dates in terms of where he thinks this event will happen. So he says that if you go 24 months from now, which is literally just two years, he says, or some amount of time, I can't exactly predict where it is, but it is possible that most developers are not coding. Okay. And I think what he says here is rather accurate considering how people interpret that comment. Okay, so he says here is that coding is just kind of like that language where we talk to computers. It's not necessarily the skill in and of itself. The executive said. He said that skill in and of itself is like, how do I innovate, how do I go build something that's interesting for my end users to use? And I think this right here actually captures what most people miss about this. When people say that, okay, software engineering is going to potentially change considering the rate of tools and advancements in the AI space and how good these systems are getting at coding related tasks. And when we look into the future, we can kind of see that, okay, this is clearly going to change the industry. Now, of course, he does say here that 24 months from now things are going to look different. And, And I think 24 months from now is not a bad estimate because 24 months from now is two years. Currently it's 2024. That would be 2026. And in 24 months from now, in two years, arguably there would have at least been potentially two more scale ups of AI models. Now, maybe there might not be two more scale ups. Of course, there are all of these things that we cannot truly predict. But I do think 24 months from now, the space might be in a completely different position. Now, with that being said, if we are in a position where 24 months from now these systems are absolutely amazing, where you can simply build products through natural language prompts, then it is possible that most developers aren't going to be coding during a time where LLMs are doing the majority of the heavy lifting. So I think that that is an accurate statement for 24 months from now. What he doesn't say and what people might take away from this is that developers are going to be useless and all of their jobs are going to be gone. That's not what he's saying. What he's saying is that most of them aren't going to be coding. Remember he said this is more of like an advisory nudge rather than a die up warning. Now, the thing here as well is that coding is essentially just a language that we actually talk to computers and how we get them to do exactly what we want. And basically, if we think about like far into the future, of course it might not be 24 months from now, it might be four years from now. But the end goal is always how do we innovate and of course, how do we actually build something that's interesting for my end user users to use? I think this is mainly the end goal for anyone who's using software. The end goal is always how can I make this, you know, product better for my users? And how can I innovate within this to make products that are actually good for my users? So I think that that is a really important prediction. Now, one of the things that was recently said on Twitter, you know, earlier last year, was the fact that the hottest new programming language is English. Now this, you know, saying that the hottest new programming language is English is basically referring to the fact that English is what people use to talk to LLMs. And if you've been talking to LLMs, you'll know that when you talk to LLMs, they can manage to get a lot of your understanding through natural language. You know, sometimes you do have to do a bit of extra prompting, but as long as you understand English, you're going to be working in a very easy environment with these programs. Now, the article continues to state some more things, but like this role will change and he says the role of the software developer will change. Garmin said it just means that each of us has to get more in tune with what our customers need and what the actual end thing is that we're going to try and go build. Okay, because that's going to be more and more of what work is opposed to as of sitting down and actually writing code. And this is something that I do agree with. Once the AI is able to completely automate code, let's say, you know, 10 years from now, AI is just able to generate code for like, you know, an entire program with one single prompt, which I think we're starting to see, you know, early sparks of that with Claude 3.5. I think, of course, the main thing, you know, the kind of place that you want to be in is one way you can actually think about what the end user experience is going to be like and what customers are actually going to want. Of course it's not going to be in, you know, doing the heavy lifting through code. If LLMs are going to be doing that, the work is going to shift to be, you know, actually understanding what customers actually need and what the end thing is. So I think what he's stating here is that, you know, this role is probably not going to go away. But, you know, when you actually think about it, the role is going to change and it's going to be really interesting to see how the role manages to change when a large portion of your work does get automated. I think it's going to be interesting to see how individuals manage to adapt to that changing work environment and use any other skills in order to adapt to the workplace. Now, I do think that this is going to be something that is quite true. And of course he says here that this is no dire warning. Of course the talk of AI changing and even eliminating jobs has intensified lately as companies lay off employees or stop hiring to shift resources towards AI development. New AI tools should automatically generate code, can help companies do more with the same number of engineers or fewer of these pricey employees. If you aren't familiar with the price of a software engineer, these guys are paid big, big bucks, especially at FAANG companies and demand a higher salary than most traditional roles. You can see here it says in Garmin's case he was sharing advice rather than issuing a dire warning that software developers will go extinct because of AI. His tone is optimistic, suggesting more creative opportunities for developers. And he says that aws, Amazon Web Services was helping employees continue to upskill and learn about new technologies to to increase their productivity with the help of AI. Now, I think this is a stellar statement because there is a lot of fear right now, the fact that AI can do a lot of things and it's advancing so rapidly. The fact that this could eliminate jobs is certainly a fear amongst many, which is why of course, I do have my community. But this is something that, like I said before, is a role that I think it's going to be enhanced by AI. Because the thing is that right now what we're seeing is we're seeing an influx of people that are getting into code because of these LLMs and these systems. What we have now is a place where you can ask an LLM to code something for you, completely basic. But if you don't understand how that code works, how you can change that code, what to kind of prompt the LLM, you're still going to be pretty stuck in a rudimentary manner when you're trying to build something. And I think that is also going to actually short term increase a demand for software engineers because there are many people I know right now, including myself, who are building certain things, you know, experimenting with code that truly haven't really done that before. So it's going to be kind of interesting to see how that dynamic manages to shift and how companies manage to integrate software engineers as rather more than coders now, I guess you could say orchestrators of pieces of software as their main focus. So you can see right here, he says being a developer in 2025 may be different than what it was as a developer in 2020. And I think this is going to be rather true as your role's main focus is probably going to shift. So that's a huge hint towards any aspiring software developer or someone who is a software developer, the kinds of things that you're going to be focusing now. Essentially, he says here that this is no more undifferentiated heavy lifting. See An Amazon Web Services spokesperson, Aisha Johnson, told Business Insider that Garment's comments conveyed opportunities for developers to accomplish more than they do today with new AI tools. He added that there was no indication he expected a decline in in the role of developer. Like I said before, these tools, ideally we do want them to do the heavy lifting, which is going to free up more time for tasks that do matter, such as actually thinking about what the end user wants, which means that overall these experiences are going to get better. Now, one of the things I do also want to talk about was the fact that whilst the statement does come from the Amazon Web Services cloud chief, him stating in a private chat, it does seem quite bad, like, oh, this company had this private chat and they were behind closed doors saying that, you know, AI could take over with coding. He's not the only person that has said this, okay, you know, stability AI, that company. The CEO Imad Mostaq also predicted in 2023 that there will be no human programmers in five years. He based this off a prediction on a few factors, including the GitHub data. So 41% of all code on GitHub is currently AI generated. And the fact that Mostaq believes that ChatGPT will be available on all mobile phones without an now this is an AI overview, so I haven't fact checked all this stuff, but the point here is that he's not the only person that has made this prediction. Other people, including Nvidia's CEO, has also iterated and spoken about this as well.

I want to say something and it's, it's going to sound completely opposite of what people feel over the course of the last 10 years, 15 years. Almost everybody who sits on the stage like this would tell you it is vital that your children learn computer science. Everybody should learn how to program. And in fact, it's almost exactly the opposite. It is our job to create computing technology such that nobody has to program and that the programming language is human. Everybody in the world is now a programmer. This is the miracle of artificial intelligence. The countries the people that understand how to solve a domain problem in digital biology or in education of young people, or in manufacturing or in farming. Those people who understand domain expertise now can utilize technology that is readily available to you. You now have a computer that will do what you tell it to do. It is vital that we upskill everyone. And the upskilling process I, I believe will be delightful, surprising.

So, yeah, that's Nvidia CEO talking about the future. And I think what he's stating here, a lot of people are thinking, okay, do I now switch my career? I think what he's more so talking about is like, you know, people who are just, just starting and those who are extremely young getting into the space that by the time their career matures, the area is going to be a lot different. Now, I still think you're going to need to, you know, know the fundamentals behind the scenes and that kind of stuff. But some people disagree. Now in this video, what I wanted to do is, is I wanted to keep this video as balanced as possible because I know that there is a large amount of people out there who do believe that this is something that is completely overhyped. And I did see a recent video that actually spoke about this in a lot of depth. There were two videos that I did watch and I'm going to mention those now because I always try to, you know, understand where my biases may lie. If I'm someone who has a channel in the AI space talking about the technology, there are incentives for me to exaggerate AI claims. However, I'm not that kind of person. I understand that there are nuances to things and that sometimes the technology might not be as great as it is promised. For example, we did have the Devin saga where, you know, the Internet of bugs. Four months ago, you know, a few weeks after Devin was released, he released this video. A 25 minute investigation into the first AI software engineer. Now, essentially he stated in this video that it was, you know, an upwork lie and that, you know, essentially Devin wasn't as good as they claimed. Now that's completely understandable. A lot of the times with technology demos, things aren't as good as the creators claim because they're trying to drum up hype for their product and it worked. And the thing is is that while yes, potentially Devin was a system that may have been a bit overhyped, I think the underlying message is still quite true. The tech in the space is actually not overhyped at all. And the most surprising thing about this is that the four months since the Devon system was released, you can see that this video was literally four months ago. There have actually been numerous developments in the AI software engineering space that most people haven't paid attention. Devin actually managed to grab the collective consciousness of the software engineering space on social media. But the other small incremental updates, the ones that actually matter and are slowly moving upwards, people haven't been paying attention. That's why I put here that things move pretty quickly, especially in the AI space. And I'm going to show you guys what I'm talking about. So OpenAI decided to release the software engineering bench. They introduced this on 13th August 2024, which is just around 10 days ago now, depending on when I release this video. So they said, we're releasing a human validated subset of the SWE bench that more reliably evaluates AI model's ability to solve real world software issues. So they basically said that, look, there are issues on this bench, okay? And you guys can take a look at this here. It says that, look, all right? One of the most popular evaluation suites for software engineering is the SWE bench, a benchmark for evaluating large language models abilities to solve real world software issues. Sourced from GitHub, the benchmark involves giving agents code repository and issue description and challenging them. Now listen to this. To generate a patch that resolve the problem they described by the issue. Now, coding agents have made impressive progress on the SWE bench with top scoring agents scoring 20% on the SWE bench and 43% on the SWE bench Lite according to that leaderboard as of August 5th. But here's the kicker, okay, they said that their testing identified some SWE bench tasks which may be hard or impossible to solve. So basically what they're saying here is that, look, we made a benchmark and when we looked at the one that everyone's currently benchmarking their systems on, that benchmark had some issues that were way too hard or completely impossible to solve, leading the SWE bench to systematically underestimate the models of autonomous software engineering capabilities. I'm going to say that one more time. OpenAI came out and said that our testing indicates that some SWE bench talks are impossible to solve, leading to SWE bench systematically underestimating the model's autonomous software engineering capability. Which basically means that, look, we made benchmarks and we realized that your ones were pretty impossible to solve. And you guys aren't truly realizing how capable of these models truly are. Which is an issue because if you've been paying attention to the Space. What you'll know is that there have been major, major updates. For example, with fine tuning, you can see that the actual improvement since Devon, since four months ago, this has completely doubled. Guys in the software engineering bench, you can see right here that since the Devon area, which was around 13%, you can see that that was around here during that time. In four months, performance has doubled. Okay, We've had numerous competitors come out of the works. We've had Amazon's Q developer agent get 38.8%. We've got the factory code Droid that are aiming to build an army of autonomous software agents. And then we of course, recently had Cosine Genie, which is now the state of the art model at 43.8% on that same benchmark that Devon was on that 17% benchmark. So for those who are saying that this one doesn't work and this is awful, yada yada, yada, I would love to see what people have to say about the actual improvements, because many people are now dismissing these claims and many people are now stating that, look, this thing isn't good at all. This isn't actually any kind of reliable improvement. But if we've just been paying attention to the rate of improvement here, we can see that this is absolutely incredible. Now I've made this table in Google and you can see right here that the improvements here are rather stark. We can see that literally at the start of 2024, we were at 7%. You can literally just see right here, 7%. And now we are eight months through 2024, and we are at 43.8%. And remember, Devin was only four months ago, you can see, and in the four months, you know, we've had, you know, 38, 38, 37, 36, 26, you know, things have been moving rather, rather quickly. So it's important to know that. Now, I did some, you know, not testing, but, you know, I spoke to Claude. I wanted to ask them how quickly are we going to be moving towards 90%? Because the reason I said 90% is because to go from like 40% to 90% is rather easy. And that is going to be an acceleratory period. But to get from 90% to like 99% is a lot harder. Like making gains when you're already, you know, 90% of the way there, it becomes, you know, exponentially harder to make those additional gains. So it says here that there is a significant jump from the older models below 10% to the newer ones above 20%. The improvement seems to be accelerating with More recent models showing larger gains. This is for many factors, including the fact that all these companies are now coming out of stealth, showing everyone what they've been building. And it says here that given these observations, we can make a rough estimate. The improvement from the best 2023 model, which is retrieval, augmented generation and claw 3 opus, to the best 2024 model is about a 36.8% increase in roughly eight months. This suggests an average improvement rate of about 4 point percentage points per month. To get from 43.8% to 90%, we need an additional 46.2 percentage points. At the current rate of improvement, it would take approximately just 10 months to reach 90%. However, technological progress often follows an S curve where improvements accelerate for a while, then slow down as we approach theoretical limits. And we're likely in the acceleration phase now. Considering this and the rapid recent progress, I would estimate that reaching 90% could be possible with 6 to 12 months from the latest data in the date, placing the prediction sometime between February and August 2025. So with Claude 3.5 doing the analysis here, you can see that if we actually look at the data and we extrapolate out further and say, okay, we know what's going to happen. Claude is basically predicting that, look, within 10 months it's going to be 90% of all swe tasks that are quite possible. And then Amazon's cloud chief is basically saying that, look, in the 24 months to come, things are going to be rather different. It doesn't seem that crazy when you actually break it down number by number. And this is some important things that you do have to pay attention to, especially if you're in this space now. One of the things that most people do actually forget, and I do think that this prediction isn't overestimated. This isn't like a hypey video where I'm like, oh my God, AGI in 10 months or two months. This is more of like trying to keep it factual and grounded, literally, based on the benchmarks that we've recently seen. But we've seen like 4.6 percentage points each month. Of course, some months are going to be larger, some months are going to be not as good. But I think if, you know, if we go at the same rate, it's going to be 10 months, we can say that, okay, even if it's not 10 months, even if we extrapolate and add another year onto that, there's going to be huge amounts of significant developments. Now, like I said before, there's always one thing that most people forget. And this is where I think there's going to be huge jumps made in AIs coding capabilities in the areas of these frontier models doing stuff that just most people didn't even take into account. Okay. And that's why I said the recent videos I'm talking about, you know, such as this one right here, you know, it's saying that debunking the AI software engineer, yada, yada, yada, this is terrible. Always take into account the actual benchmarks of other AI systems because that is also important. And I did watch this video from neatcode IO where he's basically explaining that, look, hype is a marketing tool and basically stating that hype is completely out of control and that this is, you know, something that could potentially not get there just yet. But I do think that what this video doesn't pay attention to is, of course, some of the stuff and of course, some of the more recent developments and some of the other papers where they actually talk about other coding stuff, which I'm going to get into in.

A most powerful marketing tool that's ever existed. Let's talk Devon AI for a second. I made a video talking about it right after the announcement. I thought the founders were super smart, but also that I'm not worried about Devin because these people are clearly fighting an uphill battle. I don't know a lot about it. The benchmarks say that this is more effective at software engineering tasks than the other LLMs. I wonder if that's because it just puts a few things together, like it has its own capabilities to, like, do research, like go on Google, browse, stack overflow and run code and execute code. And they just put those pieces together more cohesively than GPT. Obviously in a short amount of time. They didn't create their own complex LLM. I'm pretty sure they're using one of the existing LLMs. I didn't closely examine the evidence or anything like that. It seemed kind of obvious to me. A few months later, people are realizing they were fooled by the hype. Devin is extremely overrated, at least for now. If you don't believe me, maybe you'll believe this guy. He basically proved that at this point, Devin AI is more useless than a freshman CS student who. Who's one week into their first programming course.

So what I wanted to talk about, you know, after, you know, viewing Nico's video, it was definitely an insightful video. And I do think that it grounds, you know, a lot more, you know, reality in terms of what's actually happening. Because I think what we do need to pay attention to is the reality of things. You know, a lot of things are blown out in that video he talks about Tesla's been selling full self driving for years. You know, companies make claims all the time. You've got ftx, you've got Theranos. This is something that usually does happen with tech companies, but most people aren't paying attention to what these frontier companies are saying. Okay. For example, this is a paper that didn't get a lot of information data. People just basically didn't speak about this paper, but essentially this is the AlphaCode 2 technical report. Okay. And it talks about how AlphaCode was the first AI system to perform at the level of the median competitor in competitive programming at difficult reasoning tasks involving advanced math, logic and computer science. And this paper introduces AlphaCode 2, a new and enhanced system with massively improved performance. Powered by Gemini, AlphaCode 2 relies on the combination of powerful LLMs and a bespoke searching and re ranking reckoners. When evaluated on the same platform as the original AlphaCode, we found that AlphaCode 2 solved 1.7 times more problems and performed better than 85% of competition participants. Now I'm just going to ground this in what you can understand here, including myself. We can see here that basically this was done on the code forces and this was able to get the 85% out on the codeforces. So codeforces is a well known platform for competitive programming with a ranking system that categorizes participants based on their performance in the contest. The ranks include newbie, pupil, specialist, expert candidate Master Master, International Master, Grandmaster, International Grandmaster, legendary Grandmaster, and reaching 85th percentile on code forces means performing better than 85% of all competitors. And according to the data, AlphaCode 2 ranks between the expert and candidate master levels, which were quite advanced. Now, if you want to ground that in terms of how good, it could potentially show us how these systems are going to be in terms of coding abilities and what we can look at for the future. You can see here that for code forces, the expert level, which is the top 70% to 85%, participants at this level have a strong understanding of algorithms and data structures. They're proficient in solving standard competitive programming problems and are beginning to solve more challenging problems that require deeper insights or more advanced techniques. And the comparison to software engineering roles here basically says that typically focusing on learning and applying basically programming concepts, writing clean code, contributing to projects under supervision, the main problem solving required for competitive programming at the expert or candidate level is generally beyond what is expected of a junior software engineer. And this is usually where you get a mid level to senior level software engineer. And engineers at this level are expected to have strong problem solving skills, understanding algorithms and data structures, and be capable of designing efficient and scalable solutions. They might not participate in competitive programming, but they should be comfortable with similar challenges when needed. The point here guys, is that what we have is a scenario where, you know, we shouldn't discount the technology as for what we've seen recently as that every single month this is going to continually increase. One of the things that I think most people aren't paying attention to is the fact that even OpenAI have openly said that this is going to be something themselves that they are going to be tackling. For example, take a look at this clip of Sam Altman actually talking about what one of the biggest areas of improvement is likely going to be for him moving forward more to gain there a graph a couple of decades in the future be like something changed.

Yeah. Are there applications or areas you think.

Are most promising in the next 12 months?

I'm sure I'm biased just because of.

What we do here, but coding I think is a really big one. And the only reason I include that clip is because OpenAI have previously always surprised us with their capabilities and they tend to surprise us in areas that we aren't even pretty much focusing on. So the takeaways are simple, guys. What we have here is a situation where over the next 24 to 48 months we could see a complete shift in terms of the main role of software engineers. I still think that we're going to struggle to get from 90% all the way to 99%. So I think that there will be like this S curve growth where you do have a situation where right now, you know, if you look at the benchmarks we are currently during that acceleration period where, you know, you've got these systems, you know, at the start it's really hard to make progress. Then all of a sudden the jumps happen, the jumps happen. Then as you get to like 90, 95%, that's where the taper off is. So I do think that if you're like a senior level software engineer, you're someone that's really cracked. I do think that, you know, your job is probably not going to change for the most part because you're still going to have to understand how all of those systems work together and some of the most difficult tasks. But I do think that as time goes on, AI is probably going to eat the bottom area where you're going to be writing code. And as time goes on, it's going to continue to continue to continue to shift the role of many software engineers. I think one of the most interesting things from all of this is how companies are going to change in terms of their hiring process. Is someone just going to be managing AI agents? Is there going to be a new AI agent structure that allows you to manage the entire code base? You've got AI systems that are going to be working with millions and millions of context links. Either way, I think this is something that is going to really change in the next two to four years. Arguably one of the biggest changes that's going to happen. But either way, I don't think it's all doom and gloom. I just think that the role will change and I think it's going to be fascinating to see how that happens. If you did enjoy the video, don't forget to leave a like, don't forget to subscribe and I'll see you guys in the next one.