COE Showcase - Revolutionizing Software Development: The Multi-Agent AI Collaboration - Dimash Tabay

Hi, I'm Dimash, and today I want to propose a new system that might actually change how software development works from now on. So the big thing nowadays is agentic AI and LLMs, right? And they are being used by some software development projects, especially those that are small in context. They're great. I think LLMs are superhuman in intelligence and raw problem-solving ability. But what actually limits them is the context window. The reason why human programmers are much more effective than AI programmers currently isn't because we're inherently smarter than AI programmers. It's because we have a way wider context window, so to speak. We can take way more things into account. We can remember and learn from our past mistakes. And so we arrived at the conclusion that the main thing holding back these AI systems from becoming truly capable in software development is context. And the way to solve that context problem is to use a divide and conquer approach. Instead of having one agent that manages the whole codebase and that manages everything— debugging, coding, testing, backend, frontend— We use multiple agents. So what using multiple agents does is a way to divide context and divide responsibility in a way that makes the AI's problem solving more effective. So here on the diagram, you can see the multi-agent development calibration system. Uh, we have a central knowledge repository, a code architecture graph that is built during, uh, the indexing process. And basically we have a whole team instead of one development agent. We have a manager agent which directs and gives instructions to the lower-level agents. And the way it works is that the manager agent has the higher-level info. Uh, by higher level, I mean more general, more architecture, architecture-wise than implementation. It also can access business info. Aims to satisfy the user more than focus on actual technical implementation, as opposed to the specialist agents that are demonstrated here: frontend, backend, database, and DevOps agent. And another cool thing that we've come up with is the communication, communication protocols between agents. So it's like a real team. It can communicate between each other. There's an inbox and outbox system that allows for asynchronous communication without interruptions. So for example, one agent can, can send an email— well, not an actual email, but like an email to another agent so that when another agent completes its action without interruption, it can respond to that email. So more in depth on this, uh, more in depth on the workflow with this new system. So either you already have an existing code base or you have a new project idea. If you already have a code base, it goes through code indexing. It identifies the architecture and the semantical understanding of what actually your code does. And then according to a special algorithm, it composes a team and It composes a team so that each agent in a team can properly manage its context. That is effective, effective context management. If you have a new project idea, first of all, the manager agent will constantly ask you questions about like what requirements, what specific technology to use, especially if you're more on the technical side. So requirements gathering, technology selection, and then it designs an architecture more internally speaking with its specialist agents. Then it goes into a planning phase where it goes through component assignment, interface definition, and task prioritization. So it basically creates a product requirement document, the manager agent creates, and then the specialist agents can look at it and comment on it. And during this planning phase, there's intense communication between agents. Skepticism and continuous improvement of the product requirement document. It determines the feasibility of the implementation, and this way everyone's clear and the division of responsibility is clear, and the implementation phase which follows will be more effective. So here we have the classic development, integration, testing, and deployment. Here the agents communicate less rarely, but they still have the ability to communicate. If one agent needs a clarification on a part of a code base that it doesn't have by default in this context. For example, a backend developer wants to know more, uh, a little of the specifics about the frontend implementation, what data to supply, for example, and can communicate like that. Development, integration, testing, and deployment. And, uh, here you can see where human input might be needed. So First of all, when you're gathering requirements, when you're defining interfaces, and for actual deployment. So managing like cloud infrastructure. So the next thing is, how do we actually understand the code base? Actually understand what it does. And this is brand new technology, code indexing and understanding approach. So we have some raw codebase in general, right? And then we do static analysis. We just look at structure dependencies like what modules and what folders are located within what directory. Then we go into semantic analysis and the invention of LLMs allows us to actually understand and convert code to natural human-readable or machine-readable something like instructions and purpose functionality. And then it goes and combines the static semantic analysis to create a knowledge graph, which allows agents and team members within, within this system to fully really understand what is going on within the codebase. So a huge problem with single agents nowadays is that they don't really understand the codebase. So they rewrite old functionality, they remove old functionality. And this way, with division of responsibilities and with the knowledge graph, it ensures that agents do not interfere with each other's components and can continue working on their components seamlessly. So in this way, it creates an automated architecture documentation with diagrams, decision records, and component descriptions. That makes it easier for everyone to understand. So now let's look at the agent context window. What makes this special from the regular implementation of agents is the communication. What makes it a team member instead of just an, just an executor. So a specialist agent context window includes the core identity layer. Who is it? Why is it doing it? What is it responsible for? Constraints, base knowledge. Then it includes the architectural understanding layer that we just discussed. The system design, component interaction, patterns. What is it? What specific part of the code base is it managing? Component-specific knowledge layer. So now it actually goes into the details of the code. How is it implemented? And this knowledge is not accessible, for example, for other developers and for the manager agent to not overwhelm them with their context. It's all about context management. And those two layers can actually interact with external storage such as the code repository, the vector databases for past experience storing, documentation, shared decisions, and the knowledge base. And then it has the working memory layer. What current task is assigned to the to-do list? Each agent will have a to-do list and it also can communicate by inbox and outbox. And then it also came— and then it also supplied with the possible decisions that it can make so that the LLM at the heart of the decision layer for the, for the agent can decide what to actually do. And all of this is within the context window management system that we specially designed. So now we'll look at the agent's decision layer mechanism. So we just discussed the agent's workspace. What is, what does it have? Priorities, code knowledge, inbox, and role, right? And then it takes all that information, smartly formats that, and puts that into an LLM of choice, which, off of which it can make a very smart decision on what to actually do based on context evaluation and decision selection. So it can either go into standby, wait for input. Such as it's waiting on an email that it sent. It can reply and respond to a message that it got. It can send mail if it wants to request information for— from the manager or from peer-to-peer, like frontend to backend. And then it also has another execution layer, which is make a change, modify code, and this spins up a more classical React agent. Which is how most agents are nowadays. Like the Replit agent, it's a React agent. And the reason why making a code change is a React agent is because it doesn't need to communicate while it's making that specific code change. And then after the React agent, a subpart of the whole agent returns its success and the changes to its codebase, it obviously updates the code knowledge for everyone. It updates like the success, the to-do, and so on. So it's all an interactive ecosystem of software development. So now we've arrived at the summary. So let's just recap what makes this system the new thing in software development. So the old AI systems, they cannot manage context as well as multiple— as multi-agent collaboration system. Because like even with humans, right, we don't have one guy who's managing everything, especially if it's a large codebase. We have several specialists and we plan to apply that same thing for AI agents. And this will improve massively the effectiveness of each AI agent. So the sum of its parts will be greater than the whole, and it will actually allow for non-technical people to develop and deploy apps, or for technical people to already maintain existing database— existing code bases, and just overall simplify life. And also with the multi-agent development collaboration system, it's not only communications, not only between agent to agent, so it's not only AI to AI. Obviously, it's like a team member. It's like you got a whole dev team. So it can integrate with Slack or other project management systems. You can actually assign specific agents which manage specific parts of the codebase tasks to complete, or you can just assign something to the manager about a whole more like sweeping change and will complete it. It's completely changing the game. It's like hiring a new employee, but that employee is AI. So this is why we think that the multi-agent development collaboration system will be the new big thing in software development.