Building AI-Native Product Teams: Lessons from Automata | Stathis Eleftheriadis

Welcome and thank you for coming. My name is Stathis and I'll be sharing our journey in this kind of, I would say, sea of AI that we hear every day. And this is actually a very personal topic. Typically when I do presentations, I show our product, I show our vision, I show our roadmap. Today I'll be talking about a topic that is very close to my heart. I've been working in tech for the last 10 years, I've built 2 startups. I do think there's a topic that is around high-performing teams— how to build teams, how to build effective teams— is not talked much within leadership, within startups, within larger organizations. So when this opportunity came up, I actually started reflecting on our journey as Automata the last 9 to 12 months. And for context, I think this was actually not really a planned transition for us as a team. It's something that you should all know, obviously we have an existing product in the market, so we had a very aggressive mandate, say the last 9 months, to rebuild the product from scratch. So we had limited resources, the team was fixed. And we had to find ways to make that happen, make that roadmap a reality. In, I would say, half of the time that we had to execute that roadmap, rebuilding a product from scratch can be very, very time-consuming and very, very challenging. I've done this 2 times in the past. I'm sure some of you went through the journey, you know the pain. And these are the learnings throughout this transition for us as a team. It wasn't always like, um, very kind of planned as a kind of transition, but we learned a lot along the way. So we'll be sharing this with you, and as I said, later we can have more conversations, go deeper, learn from you as well, have a bit of a deeper dive about your experiences in this space. Um, in terms of how I see things, um, evolving over time, I'm old enough to experience at least 2 different shifts. In tech. I'm not that old, but I'm actually seeing at least the early days of the internet, but also the cloud shift. And what is actually interesting, that every time there is a big shift in tech as this, organizations rethink about how they build products, but also how they organize themselves. From the early kind of industrialization, you you had kind of reduction in physical labor. Through automation, you have reduction in repetitive work. And during the internet, the access to the information was a lot, a lot easier. With cloud, the infrastructure was a lot cheaper. And these days with AI, we talk about this kind of how we execute our work. So this is something that we all start talking a lot more. But every time this shift happens, and what has been kind of evident that there is always the organizations that start shifting their edge where they can actually get the value from. And we've seen this, for example, with, with cloud. We started talking about product management became like a field, how we actually innovate, how we build products. It became a whole stream of work. And with AI, we are at this point, and what we've observed as a team where the value actually shifts is like towards the decision-making and the judgment. Execution becomes obviously faster, but the critical parts of the job is about the judgment and decision-making, where we see the competitive edge of most teams coming to play. And that's kind of how the journey started with us. And I think most of the companies going into the easy path of execution becoming an easier journey with AI because the solutions are easier. So you can actually prototype faster, you can build faster, you can scaffold an application in days. I mean, you're here with Anthropic, the model is within a week you can actually build a quick application, which is absolutely valid. You can test the applications faster, but at the same time you can actually ship that to the customers in a record time. But what I've actually seen is that whilst this is happening, there is a side of this equation that is often overlooked, which is more like how we actually understand what exactly needs to be built. And I've been in multiple conversations where people are scared about what this AI wave will bring to us, but in our opinion, I think where the biggest kind of opportunity is how the product teams will get a better understanding of the problem space. What are the customer needs? What are the trends in the market? What are the biggest problems our customers care about? Having this opportunity to execute faster makes that part, in our opinion, even more critical. And we've seen this the last 9 months with the teams, and we can go through very soon, like, how this panned out for us. Once we realized this, actually, we stopped thinking the AI as an application. We have a set of tools that we use across different, different functions, and we start thinking more about how we work as a team and start looking at the different phases that we go through along our journey, along our product lifecycle. So looking at understanding how we create things, how we build things, how we test and validate, how we operate and deliver things, and ultimately how we enable these solutions to the customers. And these are some things that we've been doing the last few months. And I think, again, um, I don't know, depending on the level of, uh, that your teams— I think a lot of this would be, I think, relevant to you as well. But for example, when we do discoveries, um, all the information data from customer interviews, from Zoom calls, whatever tool that you use, it's been digested, analyzed, and evaluated. Understanding history from Jira tickets or whatever system, ticketing system that you use, it's been evaluated and understood. Feedback from customers, again depending on whatever tools you use, you can ingest and basically have your taxonomy and be able to analyze that and be able to tie that back to specific user problems and customer problems. Understanding around the market, getting signals from the market on a weekly basis, understanding what are the competitors doing, how do they compare against us, having specific tooling internally that makes that available to your product team, to your product marketing team, and ultimately to the team that builds the solution. On the design side, again, this is another wave that is very, very kind of well understood— how quickly you can iterate with design prototypes. You can build something almost pixel perfect, you can actually iterate very quickly, And there is a big debate around who does this in these new teams. Is it the PM? Is it the design team? Is it the engineer? Who actually does it? Looking at what the solution would look like overall, overall UX, looking at what are the key problems that our users care about, but also that becoming the starting point of how we actually produce content for our end users. So there is a whole kind of lineage around how we actually design solutions. On the build side, again, this is something that I think you are probably most familiar with. You probably used Claude Code or any other kind of AI tool to build or have this transition between design and coding, assuming that you might be using the, the shared design system. But also we use it a lot for root cause analysis. Obviously having hooked up with our backend, we'll be able to quickly evaluate where the problems might lie, and that actually has reduced a lot the bug fixing and triaging process for our team. On the testing side, most of the testing, at least on our side, is automated. And I know this might sound very basic for some of you, but obviously, as Rui said, we operate on an intersection between physical and digital space. We don't really have software, we also have hardware. We have hundreds and hundreds of different devices that need to be orchestrated with our software. So testing is actually one of the biggest problems that we have to deal with as a product team. And I've worked in SaaS for a long time, but this is the first time we actually care so much about making sure that testing works very effectively and be able to understand where the potential bottlenecks would be on the testing side, be able to predict and ideally have a very good coverage of the actual hardware testing that goes into the product. On the delivery side, again, this is something that, as I said, it's been really top priority for the team. We had to meet very strict timelines Not just to execute the roadmap, but the new product is supposed to be rolled out to, well, our biggest customer yet. So there was no pressure there at all. So we had to find ways to make sure that the release to that specific customer, it's actually timely, it is ready, and making sure that all the updates to the stakeholders and the customers are on time. And all this process has been automated, and I can talk a bit more about how this has been achieved. And lastly, the enablement side, which again, a lot of the learnings are tied to how we operate. We release the product to initially our internal teams, which is the CX teams that go to sites and deploy the solutions to the customers. And obviously, as we roll out this product, to these sites, we need to make sure that those users are enabled to basically deploy the solution on-site. And this is actually a very key part of what we've achieved the last 9 months because we had to basically keep building the product whilst we roll out this to the CX team to keep deploying the system. And all the kind of path from building the initial PRD all the way to the release notes. This is actually a journey that is end-to-end automated, and ultimately the learning and the kind of ability to train those new users into the completely new system has been completely embedded within this workflow. But this hasn't been really from day one the way we operate it, and that's actually the fun part because Uh, we kicked off this last September, uh, and there was very much like a few licenses of ChatGPT that we were— people are using and trying to play with it for themselves to increase their, their productivity. There were a few Claude users, not even Claude Code at the time. Um, it was fun. There was not really a path, there was not really a plan of what this means broadly speaking as a team. But then as we start getting more aggressive with what we wanted to achieve, we start thinking about like along the journey where could be opportunities for us to, to unlock value. And start thinking about getting access to Cloud Code, getting access to Figma Makes to produce the prototypes, documentation in Notion. Getting the tickets and the release notes automated through Jira and so on and so on. And this became like the part where we spend most of our time. I'll be honest with you, there was a lot of learnings at this point in terms of how this can be shared across the organization because people were— certain people were more proficient, certain people were less proficient. There were not really guardrails in terms of how this system would be used. But ultimately, we have entered the stage where I'm most excited about, because this is the place where actually, as we transition out from the current state of operation, this is what the new kind of setup would also include. And it's actually expanding to this, what we call the organizational capability. There's a lot of learnings around different people using the systems how this can be shared across a knowledge base. I don't know if you're familiar with the concept of a harness. You can build your own harness and use that internally. You can have shared skills, integrations with MCP, having shared workflows, and obviously agents are doing certain things on the fly. The QA process, for example, QA agents doing something in the background, it's lovely. No one cares about that, but it is actually very important, and so on and so forth. So that's exactly kind of the journey. It's been painful but also very exciting because you can actually start seeing this evolution throughout the time and the value from the team and the excitement from their side about the learnings along the way. In terms of the different functions, because I do hear at times also during interviews. How do we use AI across the different parts? Because I'm interviewing from product design, engineering, and there are certain things that we intentionally unblocked through AI. As I said, a lot of the synthesizing, the requirements creation, PRDs, etc., have been accelerated with AI. I, I've already put the competitor analysis because we do have an agent that actually does that. We actually scrape specific things, specific customers, specific trend words, and we actually analyze against our own ontology feature set within Automata. So that's actually almost automated, but where the value is, is more about understanding the customer, speaking with the customer, and going deeply into what exactly they care about. Where does the market go? We do have a massive opportunity ahead of us, very aggressive roadmap through the partnerships that we've built to move into very new spaces. How quickly we can get that understanding and how we can prioritize effectively into that journey. So that's kind of the team's goal, that's the priority. If they're not doing that, they're not doing the right work right now. Same thing on the design side. A lot of the prototyping is almost automated and it's standardized. People can actually do that very quickly now. But looking at how good is the experience and really, really spending the time to understand if the UX that we want to achieve now and in 2 years is, is hitting the mark, that's the biggest part. On the engineering side, again, I think we had multiple cycles. People love the fact that they can get the superpower of getting things out of the way quickly. But what we've actually seen is that all the critical decisions around architecture and the overall strategy on the technical side has been honestly the biggest challenge. And that's why most of the energy for the team goes in there. And ideally most of the mundane tasks should be automated, should be reduced, should be faster. On the QA side, I did mention it already so I'm not going to spend too much time on this but we do spend most of the time understanding how confident we are around the releases. And again, for context, a release that goes bad into our market It's, it's very bad. There are live samples. People can test things treating cancer or dealing with a disease, and having a release that can break the system and take the system offline for a week, this is a no-go. And on the rollout side, as I said, again, most of the time the team is actually emphasizing around the messaging. How do you communicate the value back to the users, back to the market? And how do you make sure that people understand that this is actually useful for them instead of just automating the release notes, which again, it's a low kind of impact work from our side. So there are 4 areas that I would say overlapping along our journey. Like, I'm not gonna spend too much time on it. The first one is around the context. And I think— I can't remember who I was discussing earlier about the models and trying different models or switching models, model hopping. Um, but ultimately what we've seen is that giving cloud code access to your knowledge system— could be repos, could be the design system, could be product docs, could be knowledge base That becomes the most effective way to get the best results from, from these systems. Tools— everyone has their own tools. Again, I think there is a stat around how many tools every company uses, but the idea again on our side, what worked very well, is trying to work with these tools along the way instead of just trying to say, yeah, we're gonna just consolidate and remove everything. There is specific knowledge, specific workflows embedded within each of these tools that we decided to adopt them in the workflows. Having the user in the loop has been key priority for us and one of the key pillars that we set from the beginning. Obviously, from the, the type of product that we operate in, that becomes like something really important and making sure that we as users always have the last saying in, in the process. And the last part, which is, as I mentioned, where we are at this, at this moment, is having this ability to make the system better and better every time we use it. And having this internal knowledge system that powers everything that we do going forward, that actually is the thing that we've seen working the best for us at the moment. There are 3 main learnings, if we want to call it the AI-native team, that I think we're going to carry forward into our new operating model, which is this transition from this individual productivity to having this organizational capability. And as I said, you moving from single use of GPTs or Claude into this shared skills and having shared standards across everyone. Optimizing execution to optimizing judgment. And I think this is, for me as a PM, probably the most important one, um, because it's easy to optimize throughput story points, tickets, whatever you want to call it. But ultimately, and we see it every day, people try to do POCs, vibe code things, and then they say, oh, it doesn't work. Who's gonna roll it out to production? Do you build the right product, and do you make sure they make the right product decisions? Do you have the right trade-offs? Do you make sure that you make some tough calls? So This is ultimately where we've seen the biggest kind of opportunity and learning so far throughout our journey. And the last one is this kind of easy content generation kind of trap to having this context preservation. And this is purely within our function. Obviously, we operate within product design and engineering. We do have these kind of specific steps around getting the insights from the customers, understanding the problem, designing it, building it, testing it, and rolling it out. But what we've witnessed is that every time there is this kind of handoff, which is my nemesis as a word, there is so much context that's been lost that people really, oh yeah, I didn't— I wasn't in that conversation. What is exactly the point of this? And what we actually witnessed is that by having this shared knowledge across our stack, this becomes shared context throughout the journey of the team. So having a QA engineer the context of why something is important, we've seen this becoming always a lot easier because you can understand why something is important. Or looking at design, why something has been designed that way, why certain decision was being made from an engineering point of view. So having this shared context, we've seen actually becoming like, I don't know, 10x in a way that we use AI tools, um, at the moment. And that's something we're actually investing a lot more internally. So we covered a few things. From my perspective, AI is going to be here for a while. How we use AI is gonna actually define how certain companies will win in their markets. Um, it's not just about building stuff fast, which I think is natural for most teams to fall into that kind of mindset. And it's exciting. I like building things during the weekends as well. But ultimately, in a commercial setting, in a high-performing setting, in a setting where we need to make sure that the business succeeds, what we've seen is actually how quickly we can learn and how we can make better decisions for the product overall. Thank you, guys. Thank you. So the question is specifically around the boundaries between the different functions and how blurred they are at the moment and whether we are planning on staying in that mode for foreseeable future. Um, and I was saying that yes, the shared context is actually where we are currently in. There are some individuals that might be pushing the boundaries between design and engineering, product and design, but there hasn't been, and I don't think there is going to be in the near future any major change in the model that we operate. Um, the reason being, as I said at the beginning, especially for PMs, for example, I see on LinkedIn almost every day, oh yes, I coded this. Oh yeah, thanks. Okay, so what? I'm not— if I am evaluating a PM who I'm working with, um, I'll say that's good, but can you give me what I need from you? Do we understand what the customers care about? If they can do that, perfectly fine. For me, the way I see it is in the example of PMs, it is another tool that could help you with discovery potentially, yes. But I don't personally see— and I've seen companies, oh yeah, this is my GitHub, I've done this many PRs, so thanks. Okay. I'm not buying it yet that this is where the product world will evolve. There is, because of that fact that we build faster, we actually are the bottleneck most of the times, because we need to go back to the team and say, oh yeah, what should I work next? And then there could be potentially issues there. How do you manage that? How can we quickly get to that point? But I don't necessarily see that blurred lines yet. There might be in certain companies, but again, my learning is based on how we operate and we have a certain type of product right now. Excellent question. Actually, we got our new principal PM join on Monday. And I've been hiring for PMs for different roles the last year and a half. And interesting enough, the one of the tasks that we give is the same, but the response we get back is always different. And this time has been amazing to see that all the candidates that we hired or we interviewed, basically they used AI tool to understand the market make the proposal and basically change how they present their, their findings, which is amazing to see. And in this particular case for PMs, we do spend a lot of energy understanding how they use AI because of the way we operate. Specifically for this new hire, they will likely be driving a lot of these initiatives going forward, so they need to have certain kind of opinion experience around using AI systems. Yeah, 100%, 100%. And that's what— that's why I think one of my messages was trying to move away from that as a mindset. And we— it's a battle we fight with Rui every day. It's like, what is the vanity metric you want to optimize for, right? It's like, you wanna optimized for outcome versus, oh yeah, we released this many features. At the end of the day, no one would care. Um, but that kind of ties back to the culture around the product team as a whole. We want to encourage, right, because we want the engineers to ask questions. Why should we build this? And these are my favorite conversations. I was having conversations today about specific functionality and the engineers were pushing back. But why should we do this now? And I love that because it makes you feel that you are in that mindset of, yeah, let's drive the value instead of, let's build another feature. But it's not easy. I know a lot of— I've worked in different types of organizations. I know it is difficult to get to that mindset, but it is— it can become difficult these days with AI. Stakeholders think, yeah, because we have AI, we can build everything faster, which in a way it's true. But as I said, like, looking at the— what is the value or the metric you want to optimize? Because ultimately you can have 100 POCs and then nothing is actually moving the needle. This hasn't been a decision just for our team, so there is a business-wide strategy to enable AI across every team. So this is in line with that kind of journey. We set specific guardrails, specific operating procedures, how we can actually use AI, and making sure that it is used consistently across. At the moment, because we are the heaviest users of AI within the, the company, we maintain the system. Probably it's not going to be the case forever, but it is going to be something that we're still going to bear for a while. As I said at the beginning, it kind of emerged as a need because we had to find a way to accelerate development and accelerate the whole process because we had to basically build the roadmap in half the time. So we kind of ended up using this initially as an experiment, and then over time it evolved because actually it was very effective. But at the same time, we're not going to talk about today, but yes, there is AI functionality that we built in the product for our end users But that's kind of a separate conversation. I think Amihai will talk a bit more about that direction later on. Yeah, I mean, from the entire end-to-end, as I said, like defining like what is the problem that we want to solve, all the way to designing it, building it, testing it. So the entire flow, this is basically a very realistic example of how we go through most of the functionality that we need to build. But it is a continuum, which is the beauty of it, because in the past we started predominantly with engineers who say, oh yeah, I can use Copilot to build something faster. But then there is always a bottleneck in terms of, oh yeah, I can use that, but then what else? Recently I was just talking with the design team, for example. We're actually thinking of creating an agent that will become our user persona so we can do validation of designs faster. Because again, I think a message I want to pass is that you need to understand your market and your space because obviously we don't have access to hundreds and thousands of users. Right, so going through validation of a design is always a bit difficult and slow. Yeah. So how can we accelerate this? We understand our user persona, we can create an agent and use it as a way to validate designs. So that becomes a win-win because we can make designs, and that's kind of massive jump in decision-making from a design team because typically they go with hypothesis. And then we test internally, then we potentially have a customer, but it's so slow. So that's an example, for instance.