Webinar: Autonomous Innovation

All right, we'll get started. Welcome everyone. It's so nice to see people joining from all over the world for our webinar on autonomous innovation. Today I'm joined by Jeff Gibbons in New York. He's our Managing Director for Americas and also an enthusiast around AI powered innovation. And as a quick intro myself, I'm founder and currently Global CEO of Board of Innovation. And if you don't know us yet, we're a global innovation firm working with the global Fortune 500 on imagining and creating tomorrow's products and services. And from all of that work today we want to give you our take on how we see that evolving with AI and and give you our perspective on the future of innovation with AI. So unlike the previous webinar which you might have joined, which was very practical and tactical, this will be more conceptual and high level talking about where things could go. And then the next webinar, already giving you a sneak peek, will be more tactical again around specifically synthetic research and testing. But so for today, our take on on autonomous innovation. If you have any questions, feel free to drop them in the chat so we can answer them when we have the time. So the story for today is that we'll introduce you to what autonomous innovation is and why we believe that is where things are headed. We're going to look at some of the foundational elements of building towards new innovation engine in that scenario and give you some thoughts on where to start shaping your strategy around all of this. And we'll close it off with an invitation and also with a Q and a session when we have the time available at the end of the session. And there will also be some interactive moments during the session. So I hope you'll all stay tuned and be with us for this hour. So first of all want to introduce you to autonomous innovation, which as I stated is our perspective on where we see things going with AI powered innovation. Before we get to that, it's important to understand that all of this is driven by the rapid evolution of artificial intelligence, moving beyond machine learning and deep learning of the the past decade, moving into generative AI in the past year. AI that generates new content like text with ChatGPT, Cloud and others, images with the likes of Adobe Firefly, Midjourney and others, as well as whole virtual worlds with the likes of Run Labs and others as well. But it's likely not to stop here and evolve further into more autonomous AI, or as the co founder of DeepMind called it last week, interactive AI, which is basically AI models that operate more autonomously to accomplish goals so that move beyond currently prompting the likes of ChatGPT and getting a single answer back. AI models will be capable of interacting with different software systems with your in house data pools, interacting with people. Even so they can accomplish a more holistic task and get you the result of that whole task rather than just a Q and A basically. And just to show you like some early example of more autonomous AI model, just want to quickly show you a quick demo of Agents API which is an early a prototype of an autonomous AI agent. So I created one called Research GPT and I asked it to perform market research on the global top five non alcoholic beers. And then you start, you see it generates its own tasks, for example, retrieving the latest sales data, conducting a survey amongst non alcoholic beer consumers, analyze customer reviews from Amazon and the likes, and then it starts executing on the tasks that itself generated based on the goal that I set for the autonomous AI agent. And it starts to execute the task, learns from the tasks and then adapts and moves along in trying to complete the whole goal. At the moment these systems are very much in beta version. They obviously don't have access to your proprietary data sets yet, not to all of the market research you wanted to have. But it shows the potential of what we talk about when we're mentioning the next step beyond generative AI and more autonomous interactive AI. And that underpins our perspective on where we see innovation evolving as well. Now let's have a look at what typically happens at first when a new technology emerges. So typically what happens is that we copy what was already there and we copy that straight on the new technology platform, almost as is as an analogy, like the first time we moved newspapers on an online format or on mobile was basically a PDF version of the paper version of the newspaper. And only after a while we start shaping what could be with the new technology, basically starting from scratch and looking at which new opportunities the new technology unlocks and creating personalized news streams, suggested news articles based on what your network is reading, aggregating news articles across platforms and news sources, and all of those things that we currently have. Now when we look at the emergence of generative AI and autonomous AI, what we're seeing is that so far we've mostly done the same in leveraging those models for innovation. So what we've mostly done is looked at the current tasks that we are doing within the innovation process. For example, we're running customer interviews. So let's ask ChatGPT to draft an interview script for me. So we're basically improving or automating the current interaction innovation tasks with AI. Whereas the interesting question is to move beyond that and ask ourselves the question, how might we shape innovation in an entirely new and better way with AI almost starting from scratch? And for that I think it's important to always keep in mind innovations. Real job to be done if you want. And that isn't innovation's job to be done isn't to run 100 customer interviews or to create five prototypes. If you zoom out and look at the overall, the holistic A to B From a corporate CEO's perspective, that is to be a reliable engine for growth. By imagining creating and launching new products that are successful on the market and that become an integral and important part of a company's portfolio. That is the overall job to be done. And whether we perform 100 customer interviews for that or we create a thousand prototypes, those are just intermediary steps in the current innovation process. But maybe in the future we don't need some of those steps and we can do that in a totally different way. I think that's the mindset that we should be adopting into creating what's next for innovation itself. So let's take it a step forward into what innovation could be when powered by AI and just give you a flavor of what we see as possible. So we would see companies that are able to serve customers needs before you can actually formulate them, to predict needs and act on them before they might even happen. Concepts being generated and and rapidly assessed across desirability, feasibility, viability, sustainability and market timing. All too often we just concepts are being generated consumer led, but not necessarily taking into account feasibility and viability. In those early stages we might see models where new scientific breakthroughs, if you plug that data pool into your concept generation engine that are automatically translated to new product concepts for your consideration. If you're for example a food and beverage company and there's a new scientific breakthrough that gets published around certain types of nutrition or ingredients that would improve people's sleeps, wouldn't it be great if you would be notified of that and the system would automatically take those new scientific breakthroughs and actually come with product suggestions that would fit your brands, your portfolio and that you could consider moving forward with. We also expect to see the emergence of products and services that evolve along their life cycle so that they're more in line with the latest customer needs. And last but not least, we would see the current like synthetic testing against synthetic consumers and we will come back to that in a minute into synthetic Worlds that are more holistically modeling human behavior to test new products and services within that as a digital twin would do. So a couple of options of where things could go and overall, if we look at it from a methodology perspective, we see a historic development in terms of innovation methodology. From design thinking in the 90s which pioneered human centered design methods based on empathy and creativity. Lean startup in the 2000 and tens which pioneered and introduced much more iterative hypothesis led experimentation and validation cycles to what we call autonomous innovation, which is the AI powered imagination creating and scaling of new products. Obviously we're two steps into a marathon race, so we're only at the very early days of where this could go and it will take us a couple of years to build towards more effective innovation engines. When we look at the current AI models and having run many of our projects with clients as well, experimenting, pioneering those new systems, the key main or the key benefits that we see for innovation are moving faster, moving faster from A to B and in the end leading to a faster go to market, creating more. You can exponentially diverge in ideation, but also in the amounts of product concepts that you can move forward with much more than we normally can. And obviously also importantly, we are confident that we could increase the product success rate in the market by having AI models create, test and simulate new concepts better. And here I just wanted to open up the floor and get your input as well on how you would prioritize those benefits and what would be your priority from the tree. So if I could encourage you to just take your mobile phone and just scan the QR code that's on the screen and it will lead you directly to this question and you can just vote on one of the three options. And I'm seeing the the first votes coming in. Thank you for participating. So I'm just going to give this a couple of seconds. All right, answers are obviously still coming in and thanks for submitting them. You can keep on doing so, but in interestingly we have almost a tie for the number one spot between increasing the product success rate on the market and moving faster. So interesting to see that you're primarily leaning towards the success in market and the speed much more than actually the allowance of more divergence and creating more products than we could moving on. So what we are currently seeing is the emergence of a new AI powered innovation engine. And for us that consists of three key elements within that generate, validate, curate and maybe just to run through a really quick simple example, let's stick with the food and beverage company. And let's say you want to create new beverage products. So you would generate new beverage ideas with the current LLMs based on data that you feed it with, or data that's already there, and you prompt it based on your demand spaces, brands, etc. You would move along with quickly validating that with synthetic consumers, which we'll come back to, and you would humanly curate that with the best ideas. And for us it's important that there is still a human curation role as part of that process, either on a model level or specifically at steps in the process. And then you would start the next cycle based on the prioritized ideas. You would start generating visual designs of those. You would validate the visual pack designs, for example, with consumers you would curate again, generate and basically you would increase the levels of fidelity each cycle you you run through with this. Having looked at that, what we could see it evolved to more in the future is towards more autonomous innovation where that curation role becomes more minimal or looks at it more on on a model level where we generate and validate in really quick cycles. And maybe it's interesting to look at an edge case. And as an example, so we're looking at Shein here, which is the obviously controversial Chinese fashion brand. But if you look at them as an edge case, they are pretty interesting to learn from as well. So Shein launches a thousand new products each day. So just take that in for a second and compare that to the amount of new products that your company introduces maybe per year. And then consider that Shein launches a thousand new products per day. How do they do that? In obviously simplified ways. So they generate a thousand new fashion items per day. They generate that based on social listening as one of the data inputs. So for example, if there's something trending, a celebrity that did something with red bananas, the system would generate T shirts with red bananas on there. It would immediately push it into the web shop and then it would get validated and decided upon whether it could be killed or scaled. And probably out of the thousand they would kill 900 that don't get any traction. And then the other hundred would be scaled up obviously on the back end of this, to make this possible, they have shortened the time from concept generation to full scale production and distribution, which they all do centrally, mostly from one location to just three days. So they auto generate a new product concept, put it in the web store, and after three days it's produced and shipped around the world. As I said, it's a controversial and edge Case they had product launches that gone wrong as well. Most definitely. But it's interesting to look at an edge case of what this would be in terms of a more autonomous innovation cycle. But also you could easily see this at play in digital services. For example, in music, you already have a pretty good song or AI song generators. You could have a model that generates a thousand new songs per day, puts them into or exposes them in the Spotify algorithm to a small subset of people, and then kills or scales them, depending on whether they get any traction. And you can have that iterative cycle. I think if you look at those models, they kind of start from the basic assumption that more quantity in the end will lead to better quality or better product market success. So looking at some of those examples, I think our general point of view is that AI unlocks the potential for innovation to be not just constant, but also more autonomous. We are not advocating for systems to necessarily become fully autonomous. This could be components of your innovation engine, but always with that human curation role obviously underpinning all of it. If we move from that take on autonomous innovation, let's have a look at how do we start building, building a new innovation engine around this. First of all, it's important to understand the unique capabilities that AI models currently have. And we're seeing having them unique capabilities for synthesis, ideation and modeling human behavior. And we're not only seeing that empirically in the projects we run, but also in the studies that are being published and that we're closely following. So, for example, there was a study, there's been a couple of studies on using LLMs for ID generation. I think across all of them, the general pattern is that GPT generated ideas are at least equally valuable and often as creative as humanly generated ideas and concepts. And this specific study even concluded that GPT4 generated ideas were actually rated higher in terms of purchase intent from customers than the humanly generated ideas. So in terms of ID generation, it has unique capabilities, but also in modeling human behavior. So we've had comparative studies against asking synthetically modeled people or personalities and benchmarking that across populations and actual people. And almost all of the studies have concluded that the results you're getting are valid. And so it's another unique ability of AI models in terms of modeling human behavior. For us, there is no time to wait to start building your new innovation engine. As we see the first movers already leveraging AI to create new products and services. For example, last week you might have seen the new is on Coca Cola unleashing their Y3000, which is a soda that kind of, or that at least pitches to taste like the year 3000, and which was in part autonomously generated with AI. We're seeing that in software development, with already about 50% of all the code of all the digital products that are being developed on GitHub being generated with GitHub Copilot, which is the AI Copilot. But we're also seeing that, for example, in healthcare, where AI is enabling the discovery of new drugs by autonomously ideating and testing potential new molecules for existing conditions. When we look at the transition that we're making or the unlocks that we're seeing from the old to the new, here are a couple of them. So up until now we've mostly been applying human imagination based on insights and often disparate data that are scattered across different documents data sets in companies, but not like uniformly accessible to people. Whereas we see that moving to turbocharged human plus AI imagination. So leveraging the ID generation capabilities that we just touched upon of the LLMs and through trusted data, like having an AI model, an LLM that has access to all of your internal data, for example, all of the insights and research data, and can bring you the synthesis of that, rather than having to shift through PDFs after PDFs. We're hopeful to move from a process which is fairly siloed, phased and still fairly slow, if you come to think of it, across the innovation journey, which is based on projects and stage gates. And at least our hope is that we will evolve towards more organic, iterative and much faster process as more and more components of the innovation process become more autonomous. And we'll talk about about some of them in, in a minute. I think at the moment, if we look across the board, products are still launched pretty infrequently and are often infrequently updated. With autonomous innovation, it would enable you to do more launches, maybe not to the extent of a Cheyenne that does a thousand product launches per day. I don't think that's the ambition that many companies would have or should have, but more frequent than we currently do. And also those products being more personalized, especially if we're talking about digital products and services. Still, when you see survey after survey from CEOs and how happy they are with innovation, you still see that there is a general unhappiness about the product success rate. Like the success rate of products being launched to market being uncertain and not reliable enough using or leveraging more autonomous innovation going forward. We're confident to be able to increase product success by having pre simulated data as well as more real time market validation. But we'll come to that in a bit as well. So when we talk about building towards an autonomous innovation engine for us that consists of multiple sub engines and just want to quickly walk you through some of those. So at first it starts with an insights engine that and this list of keywords below is obviously non exhaustive but that would for example capture all of your social listening data that would capture the scientific breakthroughs to be translated into product concepts. As we touched upon earlier, it would contain of all of your proprietary data pools around sales data, customer feedback and all of those things which would feed into a concept engine. So basically you want to have as many relevant data sources, inside sources to feed into your concept generator. We Talked about how LLMs have proven to be effective concept generators. Almost equally to human capability. We would see concept screeners being modeled into that engine. You would want to model your brand models to stick to the example of the global food and beverage company. So you would want your demand space modeled in there, your brand positionings, your target customers so that you could prompt the signal, prompt the system. For example, when you would start a new project, you could prompt it saying like hey, I'm looking for a new coffee beverage that would target morning occasions for Gen Z consumers in Vietnam around demand Space xyz and based off all of the knowledge that it would have, it would start to generate relevant concepts for that brief that is still reactive. Maybe in the future it could become more proactive if it knows your strategy that would start to self generate concepts relevant to that. Equally importantly, what we're very much working on is to upgrade the current simulation engines to validate those concepts across desirability which we touched upon feasibility and viability. And it's so important to have feasibility and viability early on in the process. And I think we all know that. But because as an organization we're often organized in silos, it's sometimes difficult to get all the points of view early on in the process. And I think LLMs and especially AI agents that you can model could could interact at every stage of the process. What if you would have an AI agent from manufacturing, one from sourcing, one from marketing, one from like any other relevant actor that you would have and they would work on each stage of a concept being developed and tackle it from from all different angles. We're already seeing those type of simulations and we're very confident they can successfully be applied in the Innovation context as well. Next to the concept engine, the simulation engine to validate, you would obviously have a build engine which would consist of generative design components, generative coding components for digital and a production system obviously. And between the concept, simulation and build engine you would have iterative cycles between them leading then to a launch and scale engine which could have elements of a more autonomous lounge components like we have seen with Cheyenne, maybe for the food and beverage company that we talked about. They could also have a web shop where they would more autonomously launched beverage concepts and see which traction they they would get. We're likely to see the emergence of more autonomous products, especially in digital products and services that would adapt to whoever is in front of them and that would personalize based on the person that's using the service and more autonomous market and go to market tactics. So this is our overview of the innovation engine that we are working on, that we're working on obviously across projects and with clients as well. And what we picture we will be maturing in the coming months and years to a more autonomous innovation engine across all those sub engines. Maybe just to plug in one very concrete example, to use the example of Shein, so the Chinese fashion brand. So if you look at it across these innovation engines, so they take social listening data, the story of the red bananas, they auto generate concepts with it and they actually immediately get it to an autonomous launch and an autonomous validation in the scale engine. And on the back end of it they've perfected their production and distribution system so they have a three day turnaround from production to distribution. So this is what an extreme edge case looks like. Again, we're not advocating for others to just copy this, but it gives you an idea of what this edge case looks like mapped out across the innovation engine. Just wanted to give you the floor as well again and get your thoughts on like if you look at it across this autonomous innovation engine, where would you focus on first in terms of upgrading that sub engine towards being more autonomous, towards being more AI powered? Where do you think the biggest gains could be could be realized? If you could get your smartphones out again, that would be great. I'm really curious to see what you think. Just give it a couple of more seconds. Answers coming in. All right, let's see what we get. So well clear point of view is that you feel that the insights engine is where you would prioritize and then moving along with the concept engine and the simulation as seconds and thirds and then not focus all too much on the build engine and the scale engine. So I think in general, I guess to draw one conclusion from it, and that's also in line with our general point of view, is that we see a lot of potential to make the front end of innovation more automated and more autonomous. And you've also prioritized the more front end parts of the innovation process. Thanks a lot for sharing your thoughts here as well. When we look at launching new products to market and increasing the product market success, there are generally two roads to take. On the one hand, you could say let's just autonomously launch new products and validate them in market directly. That's the Shein model, the thousand new products per day. Or you could go the route of advanced simulation. Like what if we could evolve the current synthetic consumers and get them to higher, even higher fidelity so we can predict market behaviors and predict market success in more of a simulation type of environment. And I just want to dive a bit deeper into one component of the overall engine because we don't have the time to cover all of it. But just diving deeper into the simulation engine, luckily that was at least in your top three that you wanted to prioritize and more specifically dive into desirability simulation. So desirability simulation, A couple of angles to that, I think. An interesting one was a recent study that was published in Frontiers in Artificial Intelligence. And what they developed is a method called neuro forecasting. And they applied it here for music songs. Obviously everyone is always looking to generate the next big global hit. So they applied machine learning to neurophysiological data, which is data that you acquire by just giving people off the shelf sensors on their heads and then track how their brains respond to whatever you give them. And just on the neural activity of 33 people. So that's a pretty small subset. The model learned itself that it could predict the song's success with a 97% accuracy. So that's a lot higher because it was benchmarked against the current methods which only get to 50% accuracy. So we're close to 100% accuracy. We're just applying machine learning on the brainwave reads if you want, of 33 people. So imagine we could do this for other products and develop models around that that would go a long way into desirability simulation. So that in the end obviously what we want to prevent from doing is that we develop any products or launch anything to market that basically people are not really looking for. And that just adds to the waste and the overload of stuff we, we already have. So that's One angle neuro forecasting. Another angle is the evolution of synthetic customers. So synthetic customers for those that are unfamiliar with that is basically you use AN LLM like ChatGPT or any of the other models and you ask it to represent a certain customer, which could be a Gen Z gamer in Vietnam and you give it a couple of other parameters. And that's what I referred to earlier. Like the first studies show that it's actually that those representations are equally valid to doing that with real people. And so what I wanted to take you through is the evolving abilities of these synthetic consumers. At the moment most of them can just read and type. So this is a screenshot from a service called syntheticusers.com and so basically you can ask synthetic users or customers questions around like the frustrations they might have around the problem space you're exploring or to test concepts or ideas that you would have. The next step is that synthetic customers will start seeing and talking, which we already have on the multimodal LLMs like Bing AI and other platforms. Also we've seen that on Kepler IO which is a new service launched by two ex Google AI engineers just very recently in the last two weeks. And you can feed them brand assets for example and actually get feedback from your synthetically modeled customers. Them also seeing what you give them, not just reading in text. A next step beyond that is that synthetic consumers start behaving and acting more. So you get more complex simulations. We've seen the one from Stanford called Smallville. If you haven't seen it differently, have a look at that. Which basically they had a whole sim city with GPT modeled agents that lived like real humans for weeks with behaviors that were almost not to. Yeah, not different from what you would observe in real life on specific parameters. Also Amazon Go for example uses synthetic customers in a digital twin type of situation to simulate situations that are very rare to happen. The same we see with autonomous cars etc. So acting and behaving. But it shouldn't necessarily stop there. There was a new research published I think only last week or two weeks before, where they developed a neural network that was trained on all of the smells or at least like a very broad set of of smells. So could we in the future even have synthetic customers that start smelling and tasting which obviously could be relevant for the hypothetical food and beverage company that we've been using as an example going forward. So at the moment synthetic customers proven to be valid, but can only read and type. They start to see and talk will move into more accident behave but Potentially also into smell and taste and a lot of other sensory inputs. Maybe an important caveat we're not advocating for throughout the whole innovation process to just purely validate with synthetic customers. We see that as an important new unlock in the early stages of innovation to go a lot broader in the ideas and concepts that you generate. And you basically curate them down by synthetic testing to a smaller subset, but those we at the moment still advocate for testing them with real consumers as well, but just a smaller subset. And because of all the pre validation, obviously the confidence rate that they would be better concepts is a lot higher. So that was just one element of the engine, the desirability simulation within the simulation engine. And I think if you were to start around building a new innovation engine, you would map out your architecture in a way that we've done here and you would start working that through prioritizing, where you would start building a roadmap on how to evolve those components and working through a maturity scale, because obviously the models will still evolve very rapidly. So as a second model that kind of goes hand in hand with it, we've developed five levels of autonomy specifically for innovation. And in a way you could compare that with what happened with autonomous driving, which has been going on for decades and which today in parts of the world like San Francisco is finally coming to fruition. So it's definitely been a long journey and it will be more of a journey in terms of adoption in the coming years as well. But so similar to autonomous driving, we'll see a progression from AI powered components that offer near term benefits. Sorry, like adaptive cruise control or assisted parking in vehicles, or emergency braking and all of those things to the ultimate goal of more autonomous driving. So that's what we're seeing similar in innovation as well. There's a lot of near term benefits to be had around synthetic testing etc, while actually you're building towards your Moonshot innovation engine. So it's working on both the near term as well as the long term. Let's just quickly take the maturity model and give you one example of how that plays out for running customer interviews. And these are like the example we're giving here is actually running real customer interviews with human customers, just to specify that. So on level one, that's how we've always done it. Like if you're the service designer or the innovator in charge, you plan and conduct the interviews. Level two, AI assisted would be you ask an LLM to write the interview guide for you. AI powered would be, well, actually will get an AI chatbot to conduct the interview, but from a guide that we provided with semi autonomous would be that the AI chatbot would conduct the interview, but it goes off script based on the answers it gets and it would dive deeper into things. And autonomously would be that the AI chatbot would fully recruit, plan, conduct and analyze the customer interviews and would only give you the results, conclusions and recommendations from that. So maybe just as a practical example, a platform that we've been using is called Outset AI, which sits at the semi autonomous level. And in projects, for example, we ran over 150 interviews just over the weekend with none of our people involved with an AI chatbot that would interview customers that we trained in terms of the script, it would ask in the goals of the interview and the results have actually been pretty impressive. So that's an example of an ability that's already there that sits at the semi autonomous level Also. Quickly wanted to get your point of view on that. If you think through the different levels, really curious to hear, like, where do you think we are now on the journey to autonomous innovation? Going from human only over AI assistance, AI powered, semi autonomous to autonomous. So not on one specific task, but if you look at your whole innovation process that you are managing at your company, the whole innovation system, where do you feel that we are or that you are at the moment? Right. I'm sure we'll get more answers in, but I think there's pretty clear answer to, to the question here. So most people feel that we're at level two, which is AI assisted. And that's, that's definitely my take as well. If I, if I just look at it broadly in, in the market. And that actually ties back to what I started off with. Like typically what we do when a new technology emerges is that we keep on doing the same tasks that we've done. But we just asked the new technology to assist us with that. Remember like the PDF newspaper when it moved online. I think that's the stage that most are in at the moment. And I think if there's one message from this webinar that we want to give is to kind of think at it more holistically and how can we rethink the whole of how innovation is done in a more AI powered and in our minds in a more autonomous way, not to say that it needs to be fully autonomous. I'm also super curious about the 1% that answered autonomous. So maybe in the chat, if you feel up for it, you can elaborate on what you've built in that Space. So as a conclusion around like the second part on building a new kind of innovation engine, I think in general we believe that AI powered autonomous innovation can be a real competitive advantage for companies moving a lot faster, bringing products to market with a higher degree of success. And we just wanted to give you a starting point, shaping your own strategy around that. And for that I'm happy to hand over the floor to you, Jeff.

Excellent. Thank you, Phil. And I just wanted to take this moment, first of all, just to acknowledge that I am actually a real human, not just the chatbot, which was a question that came up in the chat. But yeah, so I think so in terms of how it becomes that competitive advantage, I think it's really important to think about how that works. A lot of the early days of people using generative AI, it was all about how do you just play around with ChatGPT using Internet data. That's really evolved when now working with companies, we're really looking at how do they actually really figure out how to use this at scale and how do they combine both the Internet data and their internal data, but also their internal capabilities. The way that we see this evolving is that companies should start out as explorers, right? So you're trying to figure out what's an experimental version of the engine that Phil just talked about, some targeted capability building, and then you want to deepen that expertise and become more of an expert, maybe just in applied select areas, but not broadly embedded, and then start to scale, right? So actually building out and integrating this into the majority of your innovation efforts, embedded in your organization and people, how people are trained, hired, who's hired, and ultimately it becomes this differentiator, right? You actually have the, you know, the entirety, maybe potentially of your innovation efforts running through this model and it's fully embedded in the way that people work. And I think it's important on that that we really see that this investment in autonomous innovation yields both near term results while getting you to that longer term vision. So it's not like you have to wait 8 years, 10 years, 20 years, as the autonomous car companies have had to do. You can actually get some results faster. You can get results very immediately by moving faster, moving broader and at lower cost, but also build towards this future advantage. And in terms of how we see companies getting there, the first piece obviously is to figure out what does this all mean for our company and what's our strategy around innovation in the new world of AI. So actually figuring out how do we actually identify what are they going to be the unlocks for us as an industry, a category, a company, a culture, a group of people, and then using that to start to identify what the right engine is for you, as Phil had talked about. But then there's two pieces that are critical to actually do anything with that. First of all is actually thinking about how do you actually put this into action through actual launches and actual projects that actually deliver. So thinking about how that can happen in a way that is faster and showing the value of AI. So these kind of demonstration projects that we've been looking at, so this is a lot of what we started with some of our clients is figuring out how to start to put it into practice and get some points on the board. And then the last piece is actually the organization and people transformation. So this is obviously, this is about the processes and systems and the people that you have today, but also who you need to be in the future. Right? So there will be lots of new jobs created by AI, contrary to what people might say, there will be masses of people who need to be hired to do model fine tuning, to be AI ethicists. And the role of innovation and strategy teams is going to evolve to really incorporate those capabilities. And so generally how we see this going is that companies need to see this as like a product that they're building, right? This engine is going to take a while to build, but it's going to start to show immediate payoffs from the very beginning. So in the first year, let's say roughly, we see that the goal is to really kind of streamline projects, improve quality, improve consistency, increase the focus on desire, on viability and feasibility, but do this within a handful of projects, prove it out, develop the business case and then also develop out the plan for actually what talent you need as well. Stage two is really looking medium term into how do you actually translate this into an at scale competitive advantage in the things you're already doing and increased innovation hit rate around the current go to market model that you have. So think of this as like Engine 2.0. So you're expanding from custom fine tuning of one or two LLMs to multiple. You're looking at actually these projects getting to launch. You're actually looking at starting to test out how this new autonomous go to market model could work. And then over the longer term you really end up having, and we don't really want to put a time frame on this because it's very hard to predict, but over the longer term you really actually have this position where you have the engine deployed and evolving. It's running all your efforts, or rather it's being run by people, but it's working autonomously. And you actually have this competitive advantage both against startups and incumbents, because you actually have the integration of everything you know and everything you could know in the future integrated into the way that you're making decisions. And so as we think about this, I guess the question is, you know, to the group and open to questions and answers on this. What is your strategy? Do you have a strategy? Is this something you are starting to think about and any of the key challenges that you're, that you're seeing. And we, you know, we would love to offer up to do these leadership sessions with folks to discuss this. You know, we have, as Phil has taken you through, we have a vision of what the future engine could look like. What are the stages to get there? What are the big questions you need to answer. And so we'd be very happy to actually set up these leadership sessions with you and your team to actually figure out what is your strategy and what's the right engine that makes the most sense for you and how you can start to put that into action. And then the last thing we wanted to get into before we have time for questions is that we have a very special invitation to a summit called the Autonomous Innovation Summit. It's going to be the first, I guess the first week of December. This is going to be the biggest global gathering in AI powered innovation. We have speakers from OpenAI, from MIT, from PepsiCo, industry, tech, academia, over 30 speakers over two days, two stages. And it will really be unpacking all of the issues that we've identified here today and many more that we didn't have time to get to as well. And so I think SINA will put into the chat the invite for the, for the summit and we'd be very delighted to see you there at that. And I guess at that point when I open up the floor to any additional questions and maybe we can address some in the chat. Yeah, there's an interesting question here around Open Innovation and how this works with that. I think it is a very, it's a very interesting opportunity to actually think about how you can expand the boundaries of your innovation ecosystem beyond just your internal data, your internal people. Yeah, I don't think we've actually really figured that out yet, but I think that's an interesting topic and certainly one we could get into more detail around in the, in the summit.

Another question maybe we can tap into is thinking about the agent GPT model, the Question is like if multiple companies are basing their insights recommendation on the same data set, like how can we identify like where competitive advantage comes from? I think that's a great question because at the moment obviously the LLMs are all trained on the same data. So basically the only thing you can get superior results with is the prompting to get better results out of it. Clearly the next step is that the models and for example also the synthetic customers that we talked about, synthetic customer panels will be trained on proprietary data. What if for the customers that you really want to target, what if you would follow like 30 people around in their daily lives or how they work around like a grocery shop if you're in that food and beverage category and actually model your synthetic consumers with much richer proprietary data based on that. We looked into the example of the neuro forecasting to predict songs and how successful they will be. What if you apply neuro forecasting and actually do some of your own research around your product categories so you build a predictive model around that. That's clearly the next step is building up these proprietary data pools and fine tuned models based based on those.

Yeah, that's really important point I think. Yeah, there will, there will not be like one LLM. I think in a given large company there might be dozens of LLMs that are trained and fine tuned in certain ways. And it's not like you're going to be interacting with that in a chatbot necessarily that might be integrated into Microsoft Teams or your spreadsheet package or whatever it might be.

Another question was what are the rules to describe a synthetic user? So the results are valuable maybe on that if you want to dig a bit deeper into that. So you have platforms that would kind of take you through that journey like synthetic users or Kepler that we mentioned. But we also often just model them on the internal LLM on the client side. We're happy to share some of the research that we have gathered around that and academic papers as well that give you more insight around what needs to hold true for those results to be valuable.

Yeah, maybe I'll take the one on simulation follow up. So if simulation is a key pillar, how is this ongoing accuracy ensured and scientifically substantiated? Yeah, it's really interesting. So maybe just quickly because I know we're coming on time, so a lot of the existing research has shown that it's actually incredibly accurate at predicting human behavior. That's actually going to get better over time though as we can actually do more actual AB testing and further refine and Improve the models based on how they respond versus actual real responses. And I think our goal now is to figure out how can they actually be better than actual what people say, because we know there are always limitations to what people will say in an interview or how they respond in a certain focus group or, or testing environment, no matter how real it is. So our goal is that we could actually even get better and maybe eliminate some of that bias.

Maybe that brings us at time. And I want to respect everyone's time as well and wrap it up here. I hope we've been able to give you a perspective on where we see the future of AI powered innovation goal, which in our minds is towards more autonomous innovation, what the drivers are behind that, the components in a future innovation engine, and some of the roadmap and maturity levels towards that. For us, this is very much the starting point of a conversation with all of you. I think as a global innovation community, we have to take on the challenge of making innovation in itself better and more affecting, using some of the new models. So that's also why we're organizing the summit, to kind of get broad perspectives and ideas in and to make this a broader conversation within the whole global innovation community. So if you have any questions from this, any builds maybe on what we have presented, any ideas or prototypes that you have worked on, it would be absolutely great if you would reach out and join the conversation around that. That's what we hope to do here. I hope this at least was a valuable starting point of that discussion for you and happy to engage on this further and obviously hoping to see you potentially at our next webinar, which will be a deep dive into synthetic research and synthetic customer testing, and on the summit in December, which will get all of those like broad and expert perspectives across the board. Thank you for today. We will follow up with the recording, the presentation, some links for all of you in an email so you'll have that as well and hope to see you next time. Thanks everyone.

Thanks everyone.